2024-09-16T08:28:46.665 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T08:28:46.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T08:28:46.762 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906962 branch: wip-guits-main-2024-09-13-1248 description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} email: gabrioux@ibm.com first_in_suite: false job_id: '7906962' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-main-2024-09-13-1248 ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - CEPHADM_STRAY_DAEMON log-only-match: - CEPHADM_ mon_bind_msgr2: false sha1: 8293d73f8690540e843a81caec373f9cc29cf705 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: 8293d73f8690540e843a81caec373f9cc29cf705 extra_packages: - cephadm selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-main-2024-09-13-1248 sha1: 8293d73f8690540e843a81caec373f9cc29cf705 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 6714 sha1: 8293d73f8690540e843a81caec373f9cc29cf705 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-main-2024-09-13-1248 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_8293d73f8690540e843a81caec373f9cc29cf705/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 8293d73f8690540e843a81caec373f9cc29cf705 targets: smithi160.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKyvyADJeBeXwh0mXl/09Sc8bbM2szPuatDG4FsC4Y+BXOyGhdOX0JqR60oIiK8COvWVIhSfSEzH5kDc5rLNQK4= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKGz7glHVUiYm6fXPLUy84CCRts5aL7l16d8dPCMeib2PaqoJaxzLmY4kg9Z+iQjmNgUULy0I6uetv/fEyIeHaM= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_07:44:22 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T08:28:46.762 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_8293d73f8690540e843a81caec373f9cc29cf705/qa; will attempt to use it 2024-09-16T08:28:46.763 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_8293d73f8690540e843a81caec373f9cc29cf705/qa/tasks 2024-09-16T08:28:46.763 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T08:28:46.765 INFO:teuthology.task.internal:Checking packages... 2024-09-16T08:28:46.791 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8293d73f8690540e843a81caec373f9cc29cf705' 2024-09-16T08:28:46.791 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T08:28:46.791 INFO:teuthology.packaging:ref: None 2024-09-16T08:28:46.791 INFO:teuthology.packaging:tag: None 2024-09-16T08:28:46.791 INFO:teuthology.packaging:branch: wip-guits-main-2024-09-13-1248 2024-09-16T08:28:46.791 INFO:teuthology.packaging:sha1: 8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:28:46.791 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-main-2024-09-13-1248 2024-09-16T08:28:46.934 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4917.g8293d73f 2024-09-16T08:28:46.936 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T08:28:46.944 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T08:28:46.945 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T08:28:46.961 INFO:teuthology.task.internal:Saving configuration 2024-09-16T08:28:46.974 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T08:28:46.986 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T08:28:47.010 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906962', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 08:23:33.497945', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKyvyADJeBeXwh0mXl/09Sc8bbM2szPuatDG4FsC4Y+BXOyGhdOX0JqR60oIiK8COvWVIhSfSEzH5kDc5rLNQK4='} 2024-09-16T08:28:47.036 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906962', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 08:23:33.499720', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKGz7glHVUiYm6fXPLUy84CCRts5aL7l16d8dPCMeib2PaqoJaxzLmY4kg9Z+iQjmNgUULy0I6uetv/fEyIeHaM='} 2024-09-16T08:28:47.036 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T08:28:47.044 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-16T08:28:47.044 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-16T08:28:47.044 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T08:28:47.112 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f570b8efbe0>, signals=[15]) 2024-09-16T08:28:47.113 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T08:28:47.362 INFO:teuthology.task.internal:Opening connections... 2024-09-16T08:28:47.363 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi160.front.sepia.ceph.com 2024-09-16T08:28:47.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:28:47.444 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2024-09-16T08:28:47.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:28:47.523 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T08:28:47.528 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2024-09-16T08:28:47.546 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2024-09-16T08:28:47.546 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2024-09-16T08:28:47.605 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream" 2024-09-16T08:28:47.605 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="9" 2024-09-16T08:28:47.605 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2024-09-16T08:28:47.605 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2024-09-16T08:28:47.605 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="9" 2024-09-16T08:28:47.605 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el9" 2024-09-16T08:28:47.605 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T08:28:47.605 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2024-09-16T08:28:47.605 INFO:teuthology.orchestra.run.smithi160.stdout:LOGO="fedora-logo-icon" 2024-09-16T08:28:47.605 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T08:28:47.606 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/" 2024-09-16T08:28:47.606 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T08:28:47.606 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T08:28:47.606 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T08:28:47.606 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2024-09-16T08:28:47.630 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2024-09-16T08:28:47.648 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2024-09-16T08:28:47.649 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-09-16T08:28:47.706 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2024-09-16T08:28:47.706 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2024-09-16T08:28:47.706 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2024-09-16T08:28:47.707 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2024-09-16T08:28:47.707 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2024-09-16T08:28:47.707 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2024-09-16T08:28:47.707 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T08:28:47.707 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2024-09-16T08:28:47.707 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2024-09-16T08:28:47.707 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T08:28:47.707 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2024-09-16T08:28:47.707 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T08:28:47.707 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T08:28:47.707 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T08:28:47.708 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-09-16T08:28:47.730 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T08:28:47.737 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T08:28:47.742 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T08:28:47.742 DEBUG:teuthology.orchestra.run.smithi160:> test '!' -e /home/ubuntu/cephtest 2024-09-16T08:28:47.744 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2024-09-16T08:28:47.765 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T08:28:47.771 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T08:28:47.771 DEBUG:teuthology.orchestra.run.smithi160:> test -z $(ls -A /var/lib/ceph) 2024-09-16T08:28:47.802 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2024-09-16T08:28:47.900 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T08:28:47.955 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T08:28:47.969 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T08:28:47.969 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T08:28:47.969 DEBUG:teuthology.orchestra.run.smithi160:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T08:28:47.970 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T08:28:47.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:28:47.987 DEBUG:teuthology.orchestra.run.smithi160:> uname -r 2024-09-16T08:28:47.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:28:47.988 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2024-09-16T08:28:48.045 INFO:teuthology.orchestra.run.smithi160.stdout:5.14.0-503.el9.x86_64 2024-09-16T08:28:48.045 INFO:teuthology.task.kernel:Running kernel on smithi160: 5.14.0-503.el9.x86_64 2024-09-16T08:28:48.046 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum install -y kernel 2024-09-16T08:28:48.047 INFO:teuthology.orchestra.run.smithi169.stdout:5.14.0-503.el9.x86_64 2024-09-16T08:28:48.047 INFO:teuthology.task.kernel:Running kernel on smithi169: 5.14.0-503.el9.x86_64 2024-09-16T08:28:48.047 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum install -y kernel 2024-09-16T08:28:51.570 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-09-16T08:28:51.636 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-09-16T08:28:53.321 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-16T08:28:53.937 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 17 MB/s | 20 MB 00:01 2024-09-16T08:28:57.488 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - CRB 18 MB/s | 6.5 MB 00:00 2024-09-16T08:28:58.872 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - Extras packages 102 kB/s | 19 kB 00:00 2024-09-16T08:28:59.488 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-16T08:28:59.768 INFO:teuthology.orchestra.run.smithi160.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-16T08:29:01.014 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 93 kB/s | 19 kB 00:00 2024-09-16T08:29:02.159 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-16T08:29:05.257 INFO:teuthology.orchestra.run.smithi160.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-09-16T08:29:07.280 INFO:teuthology.orchestra.run.smithi160.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T08:29:07.280 INFO:teuthology.orchestra.run.smithi160.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T08:29:07.370 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:29:07.375 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:29:07.375 INFO:teuthology.orchestra.run.smithi160.stdout: Package Architecture Version Repository Size 2024-09-16T08:29:07.375 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:29:07.375 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:29:07.375 INFO:teuthology.orchestra.run.smithi160.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T08:29:07.375 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-09-16T08:29:07.375 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T08:29:07.376 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T08:29:07.376 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T08:29:07.376 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:29:07.376 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:29:07.376 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:29:07.376 INFO:teuthology.orchestra.run.smithi160.stdout:Install 4 Packages 2024-09-16T08:29:07.376 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:29:07.377 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 79 M 2024-09-16T08:29:07.377 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 126 M 2024-09-16T08:29:07.377 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:29:07.517 INFO:teuthology.orchestra.run.smithi160.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 243 kB/s | 23 kB 00:00 2024-09-16T08:29:07.590 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-16T08:29:07.942 INFO:teuthology.orchestra.run.smithi160.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-09-16T08:29:08.526 INFO:teuthology.orchestra.run.smithi160.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 31 MB/s | 35 MB 00:01 2024-09-16T08:29:09.018 INFO:teuthology.orchestra.run.smithi160.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 19 MB/s | 29 MB 00:01 2024-09-16T08:29:09.018 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:29:09.018 INFO:teuthology.orchestra.run.smithi160.stdout:Total 48 MB/s | 79 MB 00:01 2024-09-16T08:29:09.613 INFO:teuthology.orchestra.run.smithi169.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T08:29:09.613 INFO:teuthology.orchestra.run.smithi169.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T08:29:09.704 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:29:09.706 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:29:09.709 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:29:09.709 INFO:teuthology.orchestra.run.smithi169.stdout: Package Architecture Version Repository Size 2024-09-16T08:29:09.709 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:29:09.709 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:29:09.709 INFO:teuthology.orchestra.run.smithi169.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T08:29:09.709 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T08:29:09.709 INFO:teuthology.orchestra.run.smithi169.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T08:29:09.709 INFO:teuthology.orchestra.run.smithi169.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T08:29:09.710 INFO:teuthology.orchestra.run.smithi169.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T08:29:09.710 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:29:09.710 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:29:09.710 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:29:09.710 INFO:teuthology.orchestra.run.smithi169.stdout:Install 4 Packages 2024-09-16T08:29:09.710 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:29:09.710 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 79 M 2024-09-16T08:29:09.711 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 126 M 2024-09-16T08:29:09.711 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:29:09.933 INFO:teuthology.orchestra.run.smithi169.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 184 kB/s | 23 kB 00:00 2024-09-16T08:29:10.050 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:29:10.050 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:29:10.217 INFO:teuthology.orchestra.run.smithi169.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-09-16T08:29:10.650 INFO:teuthology.orchestra.run.smithi169.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 41 MB/s | 35 MB 00:00 2024-09-16T08:29:10.997 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:29:10.997 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:29:11.059 INFO:teuthology.orchestra.run.smithi169.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-16T08:29:11.059 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:29:11.059 INFO:teuthology.orchestra.run.smithi169.stdout:Total 58 MB/s | 79 MB 00:01 2024-09-16T08:29:11.739 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:29:12.089 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:29:12.089 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:29:13.087 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:29:13.088 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:29:13.437 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:29:14.260 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T08:29:14.351 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:29:15.527 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:29:16.204 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T08:29:16.287 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:29:17.159 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:29:17.295 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:29:19.105 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:29:19.240 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:29:22.142 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:29:22.254 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:29:24.057 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:29:24.107 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:29:27.106 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:29:28.994 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:30:05.557 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:30:05.596 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:30:05.895 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:30:05.896 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T08:30:05.896 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:30:05.896 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:30:06.772 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:30:06.772 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:30:06.772 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:30:06.773 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T08:30:06.773 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T08:30:06.773 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T08:30:06.773 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T08:30:06.773 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:30:06.773 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:30:07.136 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T08:30:07.226 INFO:teuthology.orchestra.run.smithi160.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T08:30:07.226 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T08:30:07.226 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T08:30:07.226 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T08:30:07.226 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T08:30:07.226 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T08:30:07.227 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi160.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T08:30:07.227 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T08:30:07.285 INFO:teuthology.orchestra.run.smithi160.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T08:30:07.285 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T08:30:07.285 DEBUG:teuthology.orchestra.run.smithi160:> sudo rpm -qi grub2-tools 2024-09-16T08:30:07.318 INFO:teuthology.orchestra.run.smithi160.stdout:Name : grub2-tools 2024-09-16T08:30:07.318 INFO:teuthology.orchestra.run.smithi160.stdout:Epoch : 1 2024-09-16T08:30:07.319 INFO:teuthology.orchestra.run.smithi160.stdout:Version : 2.06 2024-09-16T08:30:07.319 INFO:teuthology.orchestra.run.smithi160.stdout:Release : 82.el9 2024-09-16T08:30:07.319 INFO:teuthology.orchestra.run.smithi160.stdout:Architecture: x86_64 2024-09-16T08:30:07.319 INFO:teuthology.orchestra.run.smithi160.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T08:30:07.319 INFO:teuthology.orchestra.run.smithi160.stdout:Group : Unspecified 2024-09-16T08:30:07.319 INFO:teuthology.orchestra.run.smithi160.stdout:Size : 8274670 2024-09-16T08:30:07.319 INFO:teuthology.orchestra.run.smithi160.stdout:License : GPLv3+ 2024-09-16T08:30:07.319 INFO:teuthology.orchestra.run.smithi160.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T08:30:07.319 INFO:teuthology.orchestra.run.smithi160.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T08:30:07.319 INFO:teuthology.orchestra.run.smithi160.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T08:30:07.319 INFO:teuthology.orchestra.run.smithi160.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T08:30:07.319 INFO:teuthology.orchestra.run.smithi160.stdout:Packager : builder@centos.org 2024-09-16T08:30:07.319 INFO:teuthology.orchestra.run.smithi160.stdout:Vendor : CentOS 2024-09-16T08:30:07.319 INFO:teuthology.orchestra.run.smithi160.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T08:30:07.320 INFO:teuthology.orchestra.run.smithi160.stdout:Summary : Support tools for GRUB. 2024-09-16T08:30:07.320 INFO:teuthology.orchestra.run.smithi160.stdout:Description : 2024-09-16T08:30:07.320 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:30:07.320 INFO:teuthology.orchestra.run.smithi160.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T08:30:07.320 INFO:teuthology.orchestra.run.smithi160.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T08:30:07.320 INFO:teuthology.orchestra.run.smithi160.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T08:30:07.320 INFO:teuthology.orchestra.run.smithi160.stdout:hardware devices. 2024-09-16T08:30:07.320 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:30:07.320 INFO:teuthology.orchestra.run.smithi160.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T08:30:07.321 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T08:30:07.321 INFO:teuthology.task.kernel:Updating grub on smithi160 to boot 5.14.0-505.el9.x86_64 2024-09-16T08:30:07.321 DEBUG:teuthology.orchestra.run.smithi160:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T08:30:07.497 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:30:07.538 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:30:07.812 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:30:07.812 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T08:30:07.812 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:30:07.812 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:30:07.939 INFO:teuthology.orchestra.run.smithi160.stderr:Generating grub configuration file ... 2024-09-16T08:30:08.702 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:30:08.702 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:30:08.702 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:30:08.702 INFO:teuthology.orchestra.run.smithi169.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T08:30:08.703 INFO:teuthology.orchestra.run.smithi169.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T08:30:08.703 INFO:teuthology.orchestra.run.smithi169.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T08:30:08.703 INFO:teuthology.orchestra.run.smithi169.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T08:30:08.703 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:30:08.703 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:30:09.041 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T08:30:09.134 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T08:30:09.134 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T08:30:09.134 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T08:30:09.134 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T08:30:09.134 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T08:30:09.134 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-16T08:30:09.134 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi169.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T08:30:09.135 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T08:30:09.193 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T08:30:09.193 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T08:30:09.194 DEBUG:teuthology.orchestra.run.smithi169:> sudo rpm -qi grub2-tools 2024-09-16T08:30:09.229 INFO:teuthology.orchestra.run.smithi169.stdout:Name : grub2-tools 2024-09-16T08:30:09.230 INFO:teuthology.orchestra.run.smithi169.stdout:Epoch : 1 2024-09-16T08:30:09.230 INFO:teuthology.orchestra.run.smithi169.stdout:Version : 2.06 2024-09-16T08:30:09.230 INFO:teuthology.orchestra.run.smithi169.stdout:Release : 82.el9 2024-09-16T08:30:09.230 INFO:teuthology.orchestra.run.smithi169.stdout:Architecture: x86_64 2024-09-16T08:30:09.230 INFO:teuthology.orchestra.run.smithi169.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T08:30:09.230 INFO:teuthology.orchestra.run.smithi169.stdout:Group : Unspecified 2024-09-16T08:30:09.230 INFO:teuthology.orchestra.run.smithi169.stdout:Size : 8274670 2024-09-16T08:30:09.230 INFO:teuthology.orchestra.run.smithi169.stdout:License : GPLv3+ 2024-09-16T08:30:09.230 INFO:teuthology.orchestra.run.smithi169.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T08:30:09.230 INFO:teuthology.orchestra.run.smithi169.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T08:30:09.230 INFO:teuthology.orchestra.run.smithi169.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T08:30:09.230 INFO:teuthology.orchestra.run.smithi169.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T08:30:09.231 INFO:teuthology.orchestra.run.smithi169.stdout:Packager : builder@centos.org 2024-09-16T08:30:09.231 INFO:teuthology.orchestra.run.smithi169.stdout:Vendor : CentOS 2024-09-16T08:30:09.231 INFO:teuthology.orchestra.run.smithi169.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T08:30:09.231 INFO:teuthology.orchestra.run.smithi169.stdout:Summary : Support tools for GRUB. 2024-09-16T08:30:09.231 INFO:teuthology.orchestra.run.smithi169.stdout:Description : 2024-09-16T08:30:09.231 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:30:09.231 INFO:teuthology.orchestra.run.smithi169.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T08:30:09.231 INFO:teuthology.orchestra.run.smithi169.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T08:30:09.231 INFO:teuthology.orchestra.run.smithi169.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T08:30:09.231 INFO:teuthology.orchestra.run.smithi169.stdout:hardware devices. 2024-09-16T08:30:09.231 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:30:09.231 INFO:teuthology.orchestra.run.smithi169.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T08:30:09.232 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T08:30:09.232 INFO:teuthology.task.kernel:Updating grub on smithi169 to boot 5.14.0-505.el9.x86_64 2024-09-16T08:30:09.232 DEBUG:teuthology.orchestra.run.smithi169:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T08:30:09.273 INFO:teuthology.orchestra.run.smithi160.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T08:30:09.306 INFO:teuthology.orchestra.run.smithi160.stderr:done 2024-09-16T08:30:09.308 DEBUG:teuthology.orchestra.run.smithi160:> mktemp 2024-09-16T08:30:09.348 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.OwHVC1Kas3 2024-09-16T08:30:09.349 DEBUG:teuthology.orchestra.run.smithi160:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.OwHVC1Kas3 2024-09-16T08:30:09.423 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.OwHVC1Kas3 2024-09-16T08:30:09.560 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.OwHVC1Kas3 is 6KB 2024-09-16T08:30:09.607 DEBUG:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.OwHVC1Kas3 2024-09-16T08:30:09.623 DEBUG:teuthology.orchestra.run.smithi160:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T08:30:09.685 INFO:teuthology.orchestra.run.smithi160.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T08:30:09.686 INFO:teuthology.orchestra.run.smithi160.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T08:30:09.686 INFO:teuthology.orchestra.run.smithi160.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T08:30:09.686 INFO:teuthology.orchestra.run.smithi160.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T08:30:09.687 DEBUG:teuthology.orchestra.run.smithi160:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T08:30:09.827 DEBUG:teuthology.orchestra.run.smithi160:> sudo shutdown -r now 2024-09-16T08:30:09.887 INFO:teuthology.orchestra.run.smithi169.stderr:Generating grub configuration file ... 2024-09-16T08:30:11.237 INFO:teuthology.orchestra.run.smithi169.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T08:30:11.275 INFO:teuthology.orchestra.run.smithi169.stderr:done 2024-09-16T08:30:11.275 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2024-09-16T08:30:11.292 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.DVxizPf5Dg 2024-09-16T08:30:11.293 DEBUG:teuthology.orchestra.run.smithi169:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.DVxizPf5Dg 2024-09-16T08:30:11.360 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.DVxizPf5Dg 2024-09-16T08:30:11.494 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.DVxizPf5Dg is 6KB 2024-09-16T08:30:11.543 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.DVxizPf5Dg 2024-09-16T08:30:11.558 DEBUG:teuthology.orchestra.run.smithi169:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T08:30:11.621 INFO:teuthology.orchestra.run.smithi169.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T08:30:11.621 INFO:teuthology.orchestra.run.smithi169.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T08:30:11.621 INFO:teuthology.orchestra.run.smithi169.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T08:30:11.621 INFO:teuthology.orchestra.run.smithi169.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T08:30:11.623 DEBUG:teuthology.orchestra.run.smithi169:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T08:30:11.784 DEBUG:teuthology.orchestra.run.smithi169:> sudo shutdown -r now 2024-09-16T08:30:39.857 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T08:30:39.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-09-16T08:30:39.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:30:41.789 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-16T08:30:41.790 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-09-16T08:30:41.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:30:58.369 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-09-16T08:31:00.157 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-09-16T08:31:07.377 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-09-16T08:31:07.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:31:09.159 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-09-16T08:31:09.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:31:10.433 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-09-16T08:31:12.221 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-09-16T08:31:22.441 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-09-16T08:31:22.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:31:24.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-09-16T08:31:24.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:31:40.861 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-09-16T08:31:55.874 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-09-16T08:31:55.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:31:56.191 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-09-16T08:31:56.636 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-09-16T08:31:56.636 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-16T08:31:56.637 DEBUG:teuthology.orchestra.run.smithi160:> uname -r 2024-09-16T08:31:56.652 INFO:teuthology.orchestra.run.smithi160.stdout:5.14.0-505.el9.x86_64 2024-09-16T08:31:56.652 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T08:31:56.652 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T08:31:56.652 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T08:31:57.652 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T08:31:57.653 DEBUG:teuthology.orchestra.run.smithi160:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T08:31:57.733 INFO:teuthology.orchestra.run.smithi160.stdout:ttyS1 2024-09-16T08:31:57.756 DEBUG:teuthology.parallel:result is None 2024-09-16T08:32:24.228 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T08:32:39.229 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-09-16T08:32:39.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:32:39.622 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-09-16T08:32:39.967 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-09-16T08:32:39.968 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-16T08:32:39.968 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2024-09-16T08:32:39.984 INFO:teuthology.orchestra.run.smithi169.stdout:5.14.0-505.el9.x86_64 2024-09-16T08:32:39.984 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T08:32:39.984 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T08:32:39.984 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T08:32:40.985 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-16T08:32:40.985 DEBUG:teuthology.orchestra.run.smithi169:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T08:32:41.068 INFO:teuthology.orchestra.run.smithi169.stdout:ttyS1 2024-09-16T08:32:41.089 DEBUG:teuthology.parallel:result is None 2024-09-16T08:32:41.090 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T08:32:41.100 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T08:32:41.100 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T08:32:41.103 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T08:32:41.122 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T08:32:41.127 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T08:32:41.135 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T08:32:41.135 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T08:32:41.163 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T08:32:41.217 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T08:32:41.226 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T08:32:41.227 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T08:32:41.229 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T08:32:41.261 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T08:32:41.268 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T08:32:41.273 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T08:32:41.279 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T08:32:41.281 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T08:32:41.287 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T08:32:41.287 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T08:32:41.324 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T08:32:41.353 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T08:32:41.362 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T08:32:41.362 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T08:32:41.394 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T08:32:41.411 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T08:32:41.494 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T08:32:41.536 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T08:32:41.614 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T08:32:41.659 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:32:41.659 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T08:32:41.723 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T08:32:41.767 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T08:32:41.826 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T08:32:41.910 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T08:32:41.953 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T08:32:41.954 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T08:32:42.021 DEBUG:teuthology.orchestra.run.smithi160:> sudo service rsyslog restart 2024-09-16T08:32:42.023 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2024-09-16T08:32:42.093 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T08:32:42.109 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T08:32:42.485 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T08:32:42.492 INFO:teuthology.task.internal:Starting timer... 2024-09-16T08:32:42.492 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T08:32:42.503 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T08:32:42.512 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-16T08:32:42.512 DEBUG:teuthology.orchestra.run.smithi160:> sudo service auditd rotate 2024-09-16T08:32:42.591 INFO:teuthology.orchestra.run.smithi160.stdout:Rotating logs: 2024-09-16T08:32:42.593 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2024-09-16T08:32:42.646 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: 2024-09-16T08:32:42.648 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T08:32:42.648 DEBUG:teuthology.orchestra.run.smithi160:> /usr/sbin/getenforce 2024-09-16T08:32:42.688 INFO:teuthology.orchestra.run.smithi160.stdout:Permissive 2024-09-16T08:32:42.688 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2024-09-16T08:32:42.719 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2024-09-16T08:32:42.720 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi160.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi169.front.sepia.ceph.com': 'permissive'} 2024-09-16T08:32:42.720 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-16T08:32:42.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:32:42.755 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-16T08:32:42.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:32:42.788 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T08:32:42.788 DEBUG:teuthology.orchestra.run.smithi160:> sudo /usr/sbin/setenforce permissive 2024-09-16T08:32:42.825 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2024-09-16T08:32:42.867 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T08:32:42.877 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-16T08:32:42.880 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-16T08:32:43.023 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T08:32:43.034 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T08:32:43.035 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi160.front.sepia.ceph.com,smithi169.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T08:36:28.223 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi160.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com')] 2024-09-16T08:36:28.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-09-16T08:36:28.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:36:28.306 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-09-16T08:36:28.384 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-09-16T08:36:28.423 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-09-16T08:36:28.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:36:28.502 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-09-16T08:36:28.581 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-09-16T08:36:28.581 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T08:36:28.591 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T08:36:28.592 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T08:36:28.592 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T08:36:28.595 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T08:36:28.595 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T08:36:28.621 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T08:36:28.636 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T08:36:28.652 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T08:36:28.659 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: ntpd: command not found 2024-09-16T08:36:28.667 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T08:36:28.670 INFO:teuthology.orchestra.run.smithi160.stdout:506 Cannot talk to daemon 2024-09-16T08:36:28.685 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T08:36:28.692 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2024-09-16T08:36:28.699 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T08:36:28.704 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2024-09-16T08:36:28.718 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T08:36:28.732 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T08:36:28.764 INFO:teuthology.orchestra.run.smithi160.stderr:bash: line 1: ntpq: command not found 2024-09-16T08:36:28.768 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T08:36:28.768 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-09-16T08:36:28.768 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:36:28.768 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:36:28.768 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:36:28.768 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:36:28.790 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-09-16T08:36:28.794 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T08:36:28.794 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-09-16T08:36:28.794 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:36:28.794 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:36:28.794 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:36:28.794 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:36:28.794 INFO:teuthology.run_tasks:Running task pexec... 2024-09-16T08:36:28.804 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-16T08:36:28.804 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi160.front.sepia.ceph.com 2024-09-16T08:36:28.804 DEBUG:teuthology.orchestra.run.smithi160:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-16T08:36:28.805 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi169.front.sepia.ceph.com 2024-09-16T08:36:28.805 DEBUG:teuthology.orchestra.run.smithi169:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-16T08:36:29.161 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:36:29.172 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:36:29.172 INFO:teuthology.orchestra.run.smithi160.stdout: Package Architecture Version Repository Size 2024-09-16T08:36:29.172 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:36:29.173 INFO:teuthology.orchestra.run.smithi160.stdout:Removing: 2024-09-16T08:36:29.173 INFO:teuthology.orchestra.run.smithi160.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-16T08:36:29.173 INFO:teuthology.orchestra.run.smithi160.stdout:Removing unused dependencies: 2024-09-16T08:36:29.173 INFO:teuthology.orchestra.run.smithi160.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-16T08:36:29.173 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:36:29.173 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:36:29.173 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:36:29.173 INFO:teuthology.orchestra.run.smithi160.stdout:Remove 2 Packages 2024-09-16T08:36:29.173 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:36:29.173 INFO:teuthology.orchestra.run.smithi160.stdout:Freed space: 5.5 M 2024-09-16T08:36:29.173 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:36:29.177 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:36:29.177 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:36:29.187 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:36:29.196 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:36:29.196 INFO:teuthology.orchestra.run.smithi169.stdout: Package Architecture Version Repository Size 2024-09-16T08:36:29.196 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:36:29.196 INFO:teuthology.orchestra.run.smithi169.stdout:Removing: 2024-09-16T08:36:29.196 INFO:teuthology.orchestra.run.smithi169.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-16T08:36:29.196 INFO:teuthology.orchestra.run.smithi169.stdout:Removing unused dependencies: 2024-09-16T08:36:29.196 INFO:teuthology.orchestra.run.smithi169.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-16T08:36:29.196 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:36:29.196 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:36:29.197 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:36:29.197 INFO:teuthology.orchestra.run.smithi169.stdout:Remove 2 Packages 2024-09-16T08:36:29.197 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:36:29.197 INFO:teuthology.orchestra.run.smithi169.stdout:Freed space: 5.5 M 2024-09-16T08:36:29.197 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:36:29.200 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:36:29.200 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:36:29.237 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:36:29.238 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:36:29.262 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:36:29.262 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:36:29.377 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:36:29.388 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:36:29.453 INFO:teuthology.orchestra.run.smithi160.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-16T08:36:29.467 INFO:teuthology.orchestra.run.smithi169.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-16T08:36:29.517 INFO:teuthology.orchestra.run.smithi160.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T08:36:29.523 INFO:teuthology.orchestra.run.smithi169.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T08:36:29.982 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T08:36:29.983 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-16T08:36:30.074 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T08:36:30.074 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-16T08:36:30.236 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-16T08:36:30.236 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:36:30.236 INFO:teuthology.orchestra.run.smithi160.stdout:Removed: 2024-09-16T08:36:30.236 INFO:teuthology.orchestra.run.smithi160.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T08:36:30.236 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:36:30.236 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:36:30.373 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-16T08:36:30.373 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:36:30.373 INFO:teuthology.orchestra.run.smithi169.stdout:Removed: 2024-09-16T08:36:30.373 INFO:teuthology.orchestra.run.smithi169.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T08:36:30.373 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:36:30.373 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:36:31.044 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:00:48 ago on Mon 16 Sep 2024 08:35:42 AM UTC. 2024-09-16T08:36:31.178 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:36:31.179 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:36:31.179 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-09-16T08:36:31.179 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:36:31.179 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:36:31.179 INFO:teuthology.orchestra.run.smithi160.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-16T08:36:31.179 INFO:teuthology.orchestra.run.smithi160.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-16T08:36:31.179 INFO:teuthology.orchestra.run.smithi160.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-16T08:36:31.179 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-09-16T08:36:31.180 INFO:teuthology.orchestra.run.smithi160.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-16T08:36:31.180 INFO:teuthology.orchestra.run.smithi160.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-16T08:36:31.180 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-16T08:36:31.180 INFO:teuthology.orchestra.run.smithi160.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-16T08:36:31.180 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:36:31.180 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:36:31.180 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:36:31.180 INFO:teuthology.orchestra.run.smithi160.stdout:Install 7 Packages 2024-09-16T08:36:31.180 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:36:31.224 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 5.2 M 2024-09-16T08:36:31.224 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 19 M 2024-09-16T08:36:31.224 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:36:31.226 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:48 ago on Mon 16 Sep 2024 08:35:42 AM UTC. 2024-09-16T08:36:31.363 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:36:31.363 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:36:31.363 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T08:36:31.363 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:36:31.364 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:36:31.364 INFO:teuthology.orchestra.run.smithi169.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-16T08:36:31.364 INFO:teuthology.orchestra.run.smithi169.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-16T08:36:31.364 INFO:teuthology.orchestra.run.smithi169.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-16T08:36:31.364 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T08:36:31.364 INFO:teuthology.orchestra.run.smithi169.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-16T08:36:31.364 INFO:teuthology.orchestra.run.smithi169.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-16T08:36:31.364 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-16T08:36:31.364 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-16T08:36:31.364 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:36:31.364 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:36:31.364 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:36:31.364 INFO:teuthology.orchestra.run.smithi169.stdout:Install 7 Packages 2024-09-16T08:36:31.365 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:36:31.374 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 5.2 M 2024-09-16T08:36:31.374 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 19 M 2024-09-16T08:36:31.374 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:36:31.788 INFO:teuthology.orchestra.run.smithi169.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 391 kB/s | 44 kB 00:00 2024-09-16T08:36:31.822 INFO:teuthology.orchestra.run.smithi169.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 719 kB/s | 106 kB 00:00 2024-09-16T08:36:31.855 INFO:teuthology.orchestra.run.smithi169.stdout:(3/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 4.9 MB/s | 906 kB 00:00 2024-09-16T08:36:31.889 INFO:teuthology.orchestra.run.smithi169.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 712 kB/s | 72 kB 00:00 2024-09-16T08:36:31.915 INFO:teuthology.orchestra.run.smithi169.stdout:(5/7): python3-kmod-0.9-32.el9.x86_64.rpm 912 kB/s | 84 kB 00:00 2024-09-16T08:36:31.948 INFO:teuthology.orchestra.run.smithi169.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 8.9 MB/s | 837 kB 00:00 2024-09-16T08:36:32.165 INFO:teuthology.orchestra.run.smithi160.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 77 kB/s | 44 kB 00:00 2024-09-16T08:36:32.173 INFO:teuthology.orchestra.run.smithi169.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 11 MB/s | 3.2 MB 00:00 2024-09-16T08:36:32.173 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:36:32.173 INFO:teuthology.orchestra.run.smithi169.stdout:Total 6.5 MB/s | 5.2 MB 00:00 2024-09-16T08:36:32.266 INFO:teuthology.orchestra.run.smithi160.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 157 kB/s | 106 kB 00:00 2024-09-16T08:36:32.283 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:36:32.297 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:36:32.297 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:36:32.516 INFO:teuthology.orchestra.run.smithi160.stdout:(3/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 977 kB/s | 906 kB 00:00 2024-09-16T08:36:32.610 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:36:32.611 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:36:32.883 INFO:teuthology.orchestra.run.smithi160.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 100 kB/s | 72 kB 00:00 2024-09-16T08:36:32.909 INFO:teuthology.orchestra.run.smithi160.stdout:(5/7): python3-kmod-0.9-32.el9.x86_64.rpm 131 kB/s | 84 kB 00:00 2024-09-16T08:36:33.087 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:36:33.117 INFO:teuthology.orchestra.run.smithi160.stdout:(6/7): runc-1.1.13-4.el9.x86_64.rpm 14 MB/s | 3.2 MB 00:00 2024-09-16T08:36:33.184 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-16T08:36:33.218 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-16T08:36:33.247 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-16T08:36:33.400 INFO:teuthology.orchestra.run.smithi160.stdout:(7/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 949 kB/s | 837 kB 00:00 2024-09-16T08:36:33.401 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:36:33.401 INFO:teuthology.orchestra.run.smithi160.stdout:Total 2.4 MB/s | 5.2 MB 00:02 2024-09-16T08:36:33.455 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-16T08:36:33.486 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-16T08:36:33.512 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:36:33.528 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:36:33.528 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:36:33.848 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:36:33.849 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:36:33.916 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-16T08:36:33.959 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-16T08:36:34.067 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-16T08:36:34.126 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T08:36:34.312 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:36:34.420 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-16T08:36:34.454 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-16T08:36:34.483 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-16T08:36:34.692 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-16T08:36:34.722 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-16T08:36:35.167 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-16T08:36:35.174 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T08:36:35.174 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-16T08:36:35.174 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-16T08:36:35.175 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-16T08:36:35.175 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-16T08:36:35.175 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-16T08:36:35.175 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-16T08:36:35.191 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-16T08:36:35.306 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-16T08:36:35.372 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T08:36:35.566 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T08:36:35.566 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:36:35.566 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:36:35.566 INFO:teuthology.orchestra.run.smithi169.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T08:36:35.566 INFO:teuthology.orchestra.run.smithi169.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-16T08:36:35.566 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-16T08:36:35.566 INFO:teuthology.orchestra.run.smithi169.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-16T08:36:35.566 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:36:35.566 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:36:35.986 DEBUG:teuthology.parallel:result is None 2024-09-16T08:36:36.416 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T08:36:36.416 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-16T08:36:36.416 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-16T08:36:36.416 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-16T08:36:36.416 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-16T08:36:36.416 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-16T08:36:36.416 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-16T08:36:36.754 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T08:36:36.754 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:36:36.754 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:36:36.754 INFO:teuthology.orchestra.run.smithi160.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T08:36:36.754 INFO:teuthology.orchestra.run.smithi160.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-16T08:36:36.754 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-16T08:36:36.754 INFO:teuthology.orchestra.run.smithi160.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-16T08:36:36.755 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:36:36.755 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:36:37.130 DEBUG:teuthology.parallel:result is None 2024-09-16T08:36:37.130 INFO:teuthology.run_tasks:Running task install... 2024-09-16T08:36:37.140 DEBUG:teuthology.task.install:project ceph 2024-09-16T08:36:37.140 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705'}, 'extra_packages': ['cephadm']} 2024-09-16T08:36:37.140 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705'} 2024-09-16T08:36:37.140 INFO:teuthology.task.install:Using flavor: default 2024-09-16T08:36:37.146 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T08:36:37.146 INFO:teuthology.task.install:extra packages: [] 2024-09-16T08:36:37.147 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705', 'tag': None, 'wait_for_package': False} 2024-09-16T08:36:37.147 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:36:37.150 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705', 'tag': None, 'wait_for_package': False} 2024-09-16T08:36:37.150 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:36:37.299 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/ 2024-09-16T08:36:37.299 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4917.g8293d73f 2024-09-16T08:36:37.303 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/ 2024-09-16T08:36:37.303 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4917.g8293d73f 2024-09-16T08:36:37.411 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T08:36:37.411 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T08:36:37.411 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T08:36:37.420 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T08:36:37.420 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:36:37.420 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T08:36:37.444 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T08:36:37.444 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8293d73f8690540e843a81caec373f9cc29cf705/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T08:36:37.445 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T08:36:37.446 DEBUG:teuthology.orchestra.run.smithi160:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8293d73f8690540e843a81caec373f9cc29cf705/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T08:36:37.513 DEBUG:teuthology.orchestra.run.smithi160:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T08:36:37.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T08:36:37.588 DEBUG:teuthology.orchestra.run.smithi160:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T08:36:37.593 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T08:36:37.660 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2024-09-16T08:36:37.662 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2024-09-16T08:36:37.662 INFO:teuthology.orchestra.run.smithi160.stdout:check_obsoletes = 1 2024-09-16T08:36:37.664 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum clean all 2024-09-16T08:36:37.890 INFO:teuthology.orchestra.run.smithi160.stdout:45 files removed 2024-09-16T08:36:37.908 INFO:teuthology.orchestra.run.smithi169.stdout:45 files removed 2024-09-16T08:36:37.918 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-radosgw 2024-09-16T08:36:37.936 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw 2024-09-16T08:36:38.474 INFO:teuthology.orchestra.run.smithi160.stdout:ceph packages for x86_64 311 kB/s | 84 kB 00:00 2024-09-16T08:36:38.480 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 322 kB/s | 84 kB 00:00 2024-09-16T08:36:38.772 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-16T08:36:38.774 INFO:teuthology.orchestra.run.smithi160.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-16T08:36:39.056 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-16T08:36:39.059 INFO:teuthology.orchestra.run.smithi160.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-16T08:36:41.897 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.2 MB 00:02 2024-09-16T08:36:42.900 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.2 MB 00:03 2024-09-16T08:37:00.563 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 20 MB 00:17 2024-09-16T08:37:00.908 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 20 MB 00:16 2024-09-16T08:37:05.181 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-16T08:37:05.383 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.5 MB 00:00 2024-09-16T08:37:06.765 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 19 kB 00:00 2024-09-16T08:37:06.834 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - Extras packages 82 kB/s | 19 kB 00:00 2024-09-16T08:37:07.777 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-16T08:37:07.838 INFO:teuthology.orchestra.run.smithi160.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-16T08:37:13.186 INFO:teuthology.orchestra.run.smithi160.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T08:37:13.416 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-16T08:37:15.600 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:37:15.602 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:37:15.602 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-09-16T08:37:15.602 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:37:15.602 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:37:15.602 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 11 M 2024-09-16T08:37:15.602 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrading: 2024-09-16T08:37:15.602 INFO:teuthology.orchestra.run.smithi160.stdout: librados2 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 3.5 M 2024-09-16T08:37:15.602 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 3.2 M 2024-09-16T08:37:15.602 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-09-16T08:37:15.603 INFO:teuthology.orchestra.run.smithi160.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T08:37:15.603 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 5.5 M 2024-09-16T08:37:15.603 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 22 M 2024-09-16T08:37:15.603 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 25 k 2024-09-16T08:37:15.603 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T08:37:15.603 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T08:37:15.603 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T08:37:15.603 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 755 k 2024-09-16T08:37:15.603 INFO:teuthology.orchestra.run.smithi160.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-16T08:37:15.603 INFO:teuthology.orchestra.run.smithi160.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T08:37:15.603 INFO:teuthology.orchestra.run.smithi160.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T08:37:15.604 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T08:37:15.604 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 508 k 2024-09-16T08:37:15.604 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T08:37:15.604 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 5.5 M 2024-09-16T08:37:15.604 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T08:37:15.604 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T08:37:15.604 INFO:teuthology.orchestra.run.smithi160.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T08:37:15.604 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 45 k 2024-09-16T08:37:15.604 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 143 k 2024-09-16T08:37:15.604 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 161 k 2024-09-16T08:37:15.604 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T08:37:15.604 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 320 k 2024-09-16T08:37:15.605 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 306 k 2024-09-16T08:37:15.605 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 99 k 2024-09-16T08:37:15.605 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T08:37:15.605 INFO:teuthology.orchestra.run.smithi160.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T08:37:15.605 INFO:teuthology.orchestra.run.smithi160.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T08:37:15.605 INFO:teuthology.orchestra.run.smithi160.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T08:37:15.605 INFO:teuthology.orchestra.run.smithi160.stdout:Installing weak dependencies: 2024-09-16T08:37:15.605 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T08:37:15.605 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:37:15.605 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:37:15.605 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:37:15.605 INFO:teuthology.orchestra.run.smithi160.stdout:Install 31 Packages 2024-09-16T08:37:15.605 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrade 2 Packages 2024-09-16T08:37:15.606 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:37:15.606 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 62 M 2024-09-16T08:37:15.606 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:37:15.852 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:37:15.854 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:37:15.854 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T08:37:15.854 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:37:15.854 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:37:15.854 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 11 M 2024-09-16T08:37:15.854 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2024-09-16T08:37:15.854 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 3.5 M 2024-09-16T08:37:15.855 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 3.2 M 2024-09-16T08:37:15.855 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T08:37:15.855 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T08:37:15.855 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 5.5 M 2024-09-16T08:37:15.855 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 22 M 2024-09-16T08:37:15.855 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 25 k 2024-09-16T08:37:15.855 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T08:37:15.855 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T08:37:15.855 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T08:37:15.855 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 755 k 2024-09-16T08:37:15.855 INFO:teuthology.orchestra.run.smithi169.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-16T08:37:15.855 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T08:37:15.856 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T08:37:15.856 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T08:37:15.856 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 508 k 2024-09-16T08:37:15.856 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T08:37:15.856 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 5.5 M 2024-09-16T08:37:15.856 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T08:37:15.856 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T08:37:15.856 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T08:37:15.856 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 45 k 2024-09-16T08:37:15.856 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 143 k 2024-09-16T08:37:15.856 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 161 k 2024-09-16T08:37:15.856 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T08:37:15.856 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 320 k 2024-09-16T08:37:15.856 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 306 k 2024-09-16T08:37:15.856 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 99 k 2024-09-16T08:37:15.857 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T08:37:15.857 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T08:37:15.857 INFO:teuthology.orchestra.run.smithi169.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T08:37:15.857 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T08:37:15.857 INFO:teuthology.orchestra.run.smithi169.stdout:Installing weak dependencies: 2024-09-16T08:37:15.857 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T08:37:15.857 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:37:15.857 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:37:15.857 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:37:15.857 INFO:teuthology.orchestra.run.smithi169.stdout:Install 31 Packages 2024-09-16T08:37:15.857 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 2 Packages 2024-09-16T08:37:15.857 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:37:15.858 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 62 M 2024-09-16T08:37:15.858 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:37:16.179 INFO:teuthology.orchestra.run.smithi160.stdout:(1/33): ceph-base-19.3.0-4917.g8293d73f.el9.x86 11 MB/s | 5.5 MB 00:00 2024-09-16T08:37:16.220 INFO:teuthology.orchestra.run.smithi160.stdout:(2/33): ceph-selinux-19.3.0-4917.g8293d73f.el9. 610 kB/s | 25 kB 00:00 2024-09-16T08:37:16.329 INFO:teuthology.orchestra.run.smithi160.stdout:(3/33): libcephfs2-19.3.0-4917.g8293d73f.el9.x8 6.8 MB/s | 755 kB 00:00 2024-09-16T08:37:16.421 INFO:teuthology.orchestra.run.smithi160.stdout:(4/33): libradosstriper1-19.3.0-4917.g8293d73f. 5.4 MB/s | 508 kB 00:00 2024-09-16T08:37:16.847 INFO:teuthology.orchestra.run.smithi160.stdout:(5/33): ceph-common-19.3.0-4917.g8293d73f.el9.x 19 MB/s | 22 MB 00:01 2024-09-16T08:37:16.964 INFO:teuthology.orchestra.run.smithi160.stdout:(6/33): ceph-radosgw-19.3.0-4917.g8293d73f.el9. 8.6 MB/s | 11 MB 00:01 2024-09-16T08:37:16.980 INFO:teuthology.orchestra.run.smithi160.stdout:(7/33): python3-ceph-argparse-19.3.0-4917.g8293 338 kB/s | 45 kB 00:00 2024-09-16T08:37:17.006 INFO:teuthology.orchestra.run.smithi160.stdout:(8/33): python3-ceph-common-19.3.0-4917.g8293d7 3.4 MB/s | 143 kB 00:00 2024-09-16T08:37:17.073 INFO:teuthology.orchestra.run.smithi160.stdout:(9/33): librgw2-19.3.0-4917.g8293d73f.el9.x86_6 8.5 MB/s | 5.5 MB 00:00 2024-09-16T08:37:17.098 INFO:teuthology.orchestra.run.smithi160.stdout:(10/33): python3-cephfs-19.3.0-4917.g8293d73f.e 1.3 MB/s | 161 kB 00:00 2024-09-16T08:37:17.123 INFO:teuthology.orchestra.run.smithi160.stdout:(11/33): python3-rados-19.3.0-4917.g8293d73f.el 2.7 MB/s | 320 kB 00:00 2024-09-16T08:37:17.148 INFO:teuthology.orchestra.run.smithi160.stdout:(12/33): python3-rbd-19.3.0-4917.g8293d73f.el9. 4.0 MB/s | 306 kB 00:00 2024-09-16T08:37:17.174 INFO:teuthology.orchestra.run.smithi160.stdout:(13/33): python3-rgw-19.3.0-4917.g8293d73f.el9. 1.3 MB/s | 99 kB 00:00 2024-09-16T08:37:17.266 INFO:teuthology.orchestra.run.smithi160.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 753 kB/s | 107 kB 00:00 2024-09-16T08:37:17.291 INFO:teuthology.orchestra.run.smithi160.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 1.2 MB/s | 171 kB 00:00 2024-09-16T08:37:17.316 INFO:teuthology.orchestra.run.smithi160.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-09-16T08:37:17.359 INFO:teuthology.orchestra.run.smithi169.stdout:(1/33): ceph-base-19.3.0-4917.g8293d73f.el9.x86 4.1 MB/s | 5.5 MB 00:01 2024-09-16T08:37:17.401 INFO:teuthology.orchestra.run.smithi169.stdout:(2/33): ceph-selinux-19.3.0-4917.g8293d73f.el9. 606 kB/s | 25 kB 00:00 2024-09-16T08:37:17.402 INFO:teuthology.orchestra.run.smithi160.stdout:(17/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 333 kB/s | 45 kB 00:00 2024-09-16T08:37:17.433 INFO:teuthology.orchestra.run.smithi160.stdout:(18/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-16T08:37:17.459 INFO:teuthology.orchestra.run.smithi160.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 748 kB/s | 42 kB 00:00 2024-09-16T08:37:17.484 INFO:teuthology.orchestra.run.smithi160.stdout:(20/33): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-16T08:37:17.501 INFO:teuthology.orchestra.run.smithi169.stdout:(3/33): libcephfs2-19.3.0-4917.g8293d73f.el9.x8 7.4 MB/s | 755 kB 00:00 2024-09-16T08:37:17.509 INFO:teuthology.orchestra.run.smithi160.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 719 kB/s | 36 kB 00:00 2024-09-16T08:37:17.535 INFO:teuthology.orchestra.run.smithi160.stdout:(22/33): librdkafka-1.6.1-102.el9.x86_64.rpm 2.7 MB/s | 662 kB 00:00 2024-09-16T08:37:17.559 INFO:teuthology.orchestra.run.smithi160.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-16T08:37:17.585 INFO:teuthology.orchestra.run.smithi169.stdout:(4/33): libradosstriper1-19.3.0-4917.g8293d73f. 6.0 MB/s | 508 kB 00:00 2024-09-16T08:37:17.618 INFO:teuthology.orchestra.run.smithi160.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-16T08:37:17.644 INFO:teuthology.orchestra.run.smithi160.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-09-16T08:37:17.669 INFO:teuthology.orchestra.run.smithi160.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 276 kB/s | 26 kB 00:00 2024-09-16T08:37:17.694 INFO:teuthology.orchestra.run.smithi160.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-16T08:37:17.728 INFO:teuthology.orchestra.run.smithi160.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-16T08:37:17.761 INFO:teuthology.orchestra.run.smithi160.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-16T08:37:17.820 INFO:teuthology.orchestra.run.smithi160.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-16T08:37:17.895 INFO:teuthology.orchestra.run.smithi160.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-16T08:37:17.919 INFO:teuthology.orchestra.run.smithi169.stdout:(5/33): ceph-radosgw-19.3.0-4917.g8293d73f.el9. 5.8 MB/s | 11 MB 00:01 2024-09-16T08:37:18.044 INFO:teuthology.orchestra.run.smithi169.stdout:(6/33): python3-ceph-argparse-19.3.0-4917.g8293 361 kB/s | 45 kB 00:00 2024-09-16T08:37:18.104 INFO:teuthology.orchestra.run.smithi160.stdout:(32/33): librados2-19.3.0-4917.g8293d73f.el9.x8 10 MB/s | 3.5 MB 00:00 2024-09-16T08:37:18.127 INFO:teuthology.orchestra.run.smithi169.stdout:(7/33): librgw2-19.3.0-4917.g8293d73f.el9.x86_6 10 MB/s | 5.5 MB 00:00 2024-09-16T08:37:18.153 INFO:teuthology.orchestra.run.smithi169.stdout:(8/33): python3-ceph-common-19.3.0-4917.g8293d7 1.3 MB/s | 143 kB 00:00 2024-09-16T08:37:18.162 INFO:teuthology.orchestra.run.smithi160.stdout:(33/33): librbd1-19.3.0-4917.g8293d73f.el9.x86_ 9.5 MB/s | 3.2 MB 00:00 2024-09-16T08:37:18.167 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:37:18.167 INFO:teuthology.orchestra.run.smithi160.stdout:Total 24 MB/s | 62 MB 00:02 2024-09-16T08:37:18.186 INFO:teuthology.orchestra.run.smithi169.stdout:(9/33): python3-cephfs-19.3.0-4917.g8293d73f.el 2.7 MB/s | 161 kB 00:00 2024-09-16T08:37:18.212 INFO:teuthology.orchestra.run.smithi169.stdout:(10/33): python3-rados-19.3.0-4917.g8293d73f.el 5.4 MB/s | 320 kB 00:00 2024-09-16T08:37:18.237 INFO:teuthology.orchestra.run.smithi169.stdout:(11/33): python3-rbd-19.3.0-4917.g8293d73f.el9. 5.9 MB/s | 306 kB 00:00 2024-09-16T08:37:18.262 INFO:teuthology.orchestra.run.smithi169.stdout:(12/33): python3-rgw-19.3.0-4917.g8293d73f.el9. 1.9 MB/s | 99 kB 00:00 2024-09-16T08:37:18.273 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:37:18.310 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:37:18.310 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:37:18.479 INFO:teuthology.orchestra.run.smithi169.stdout:(13/33): ceph-common-19.3.0-4917.g8293d73f.el9. 8.8 MB/s | 22 MB 00:02 2024-09-16T08:37:18.809 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:37:18.809 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:37:19.080 INFO:teuthology.orchestra.run.smithi169.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 127 kB/s | 107 kB 00:00 2024-09-16T08:37:19.163 INFO:teuthology.orchestra.run.smithi169.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 189 kB/s | 171 kB 00:00 2024-09-16T08:37:19.289 INFO:teuthology.orchestra.run.smithi169.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 198 kB/s | 160 kB 00:00 2024-09-16T08:37:19.722 INFO:teuthology.orchestra.run.smithi169.stdout:(17/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 70 kB/s | 45 kB 00:00 2024-09-16T08:37:20.189 INFO:teuthology.orchestra.run.smithi169.stdout:(18/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 325 kB/s | 292 kB 00:00 2024-09-16T08:37:20.281 INFO:teuthology.orchestra.run.smithi169.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 76 kB/s | 42 kB 00:00 2024-09-16T08:37:20.287 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:37:20.390 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-16T08:37:20.418 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-16T08:37:20.451 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-16T08:37:20.473 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T08:37:20.499 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T08:37:20.590 INFO:teuthology.orchestra.run.smithi169.stdout:(20/33): librdkafka-1.6.1-102.el9.x86_64.rpm 465 kB/s | 662 kB 00:01 2024-09-16T08:37:20.687 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T08:37:20.745 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T08:37:20.848 INFO:teuthology.orchestra.run.smithi169.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 63 kB/s | 36 kB 00:00 2024-09-16T08:37:20.907 INFO:teuthology.orchestra.run.smithi169.stdout:(22/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 5.2 MB/s | 308 kB 00:00 2024-09-16T08:37:21.025 INFO:teuthology.orchestra.run.smithi169.stdout:(23/33): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-16T08:37:21.050 INFO:teuthology.orchestra.run.smithi169.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-16T08:37:21.075 INFO:teuthology.orchestra.run.smithi169.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-16T08:37:21.089 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-16T08:37:21.101 INFO:teuthology.orchestra.run.smithi169.stdout:(26/33): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-16T08:37:21.126 INFO:teuthology.orchestra.run.smithi169.stdout:(27/33): qatlib-24.02.0-1.el9.x86_64.rpm 236 kB/s | 221 kB 00:00 2024-09-16T08:37:21.168 INFO:teuthology.orchestra.run.smithi169.stdout:(28/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-16T08:37:21.194 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 7/35 2024-09-16T08:37:21.210 INFO:teuthology.orchestra.run.smithi169.stdout:(29/33): re2-20211101-20.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-16T08:37:21.252 INFO:teuthology.orchestra.run.smithi169.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-16T08:37:21.277 INFO:teuthology.orchestra.run.smithi169.stdout:(31/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 69 kB/s | 47 kB 00:00 2024-09-16T08:37:21.303 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 7/35 2024-09-16T08:37:21.394 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rados-2:19.3.0-4917.g8293d73f.el9.x86_64 8/35 2024-09-16T08:37:21.435 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 9/35 2024-09-16T08:37:21.494 INFO:teuthology.orchestra.run.smithi169.stdout:(32/33): librados2-19.3.0-4917.g8293d73f.el9.x8 12 MB/s | 3.5 MB 00:00 2024-09-16T08:37:21.500 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 9/35 2024-09-16T08:37:21.529 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-16T08:37:21.552 INFO:teuthology.orchestra.run.smithi169.stdout:(33/33): librbd1-19.3.0-4917.g8293d73f.el9.x86_ 11 MB/s | 3.2 MB 00:00 2024-09-16T08:37:21.557 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:37:21.558 INFO:teuthology.orchestra.run.smithi169.stdout:Total 11 MB/s | 62 MB 00:05 2024-09-16T08:37:21.573 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-argparse-2:19.3.0-4917.g8293d73f.el9. 11/35 2024-09-16T08:37:21.644 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cephfs-2:19.3.0-4917.g8293d73f.el9.x86_64 12/35 2024-09-16T08:37:21.673 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libradosstriper1-2:19.3.0-4917.g8293d73f.el9.x86_6 13/35 2024-09-16T08:37:21.674 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:37:21.711 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:37:21.712 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:37:21.737 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4917.g8293d73f.el9.x86_6 13/35 2024-09-16T08:37:21.771 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-16T08:37:21.888 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-16T08:37:21.922 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-16T08:37:21.974 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-16T08:37:22.004 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-16T08:37:22.033 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-16T08:37:22.075 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T08:37:22.120 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-16T08:37:22.211 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:37:22.212 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:37:22.438 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-16T08:37:22.480 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 23/35 2024-09-16T08:37:22.570 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 23/35 2024-09-16T08:37:22.601 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rbd-2:19.3.0-4917.g8293d73f.el9.x86_64 24/35 2024-09-16T08:37:22.637 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-16T08:37:23.134 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-16T08:37:23.166 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librgw2-2:19.3.0-4917.g8293d73f.el9.x86_64 27/35 2024-09-16T08:37:23.237 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librgw2-2:19.3.0-4917.g8293d73f.el9.x86_64 27/35 2024-09-16T08:37:23.307 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rgw-2:19.3.0-4917.g8293d73f.el9.x86_64 28/35 2024-09-16T08:37:23.335 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-common-2:19.3.0-4917.g8293d73f.el9.x8 29/35 2024-09-16T08:37:23.672 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:37:23.759 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-16T08:37:23.791 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-16T08:37:23.824 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-16T08:37:23.847 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T08:37:23.868 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T08:37:24.066 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T08:37:24.120 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T08:37:24.474 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-16T08:37:24.584 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 7/35 2024-09-16T08:37:24.677 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 7/35 2024-09-16T08:37:24.767 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:19.3.0-4917.g8293d73f.el9.x86_64 8/35 2024-09-16T08:37:24.794 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 9/35 2024-09-16T08:37:24.857 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 9/35 2024-09-16T08:37:24.894 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-16T08:37:24.938 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:19.3.0-4917.g8293d73f.el9. 11/35 2024-09-16T08:37:25.017 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:19.3.0-4917.g8293d73f.el9.x86_64 12/35 2024-09-16T08:37:25.046 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:19.3.0-4917.g8293d73f.el9.x86_6 13/35 2024-09-16T08:37:25.111 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4917.g8293d73f.el9.x86_6 13/35 2024-09-16T08:37:25.146 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-16T08:37:25.262 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-16T08:37:25.297 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-16T08:37:25.352 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-16T08:37:25.387 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-16T08:37:25.415 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-16T08:37:25.449 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T08:37:25.486 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-16T08:37:25.805 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-16T08:37:25.916 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 30/35 2024-09-16T08:37:25.965 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 30/35 2024-09-16T08:37:25.987 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 23/35 2024-09-16T08:37:26.078 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 23/35 2024-09-16T08:37:26.108 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:19.3.0-4917.g8293d73f.el9.x86_64 24/35 2024-09-16T08:37:26.144 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-16T08:37:26.544 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 30/35 2024-09-16T08:37:26.584 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 31/35 2024-09-16T08:37:26.651 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-16T08:37:26.677 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 31/35 2024-09-16T08:37:26.677 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T08:37:26.677 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T08:37:26.677 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:37:26.690 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:19.3.0-4917.g8293d73f.el9.x86_64 27/35 2024-09-16T08:37:26.719 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-selinux-2:19.3.0-4917.g8293d73f.el9.x86_64 32/35 2024-09-16T08:37:26.761 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:19.3.0-4917.g8293d73f.el9.x86_64 27/35 2024-09-16T08:37:26.840 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:19.3.0-4917.g8293d73f.el9.x86_64 28/35 2024-09-16T08:37:26.876 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:19.3.0-4917.g8293d73f.el9.x8 29/35 2024-09-16T08:37:29.164 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 30/35 2024-09-16T08:37:29.197 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 30/35 2024-09-16T08:37:29.783 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 30/35 2024-09-16T08:37:29.825 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 31/35 2024-09-16T08:37:29.917 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 31/35 2024-09-16T08:37:29.917 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T08:37:29.917 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T08:37:29.917 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:37:29.968 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:19.3.0-4917.g8293d73f.el9.x86_64 32/35 2024-09-16T08:37:36.826 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4917.g8293d73f.el9.x86_64 32/35 2024-09-16T08:37:36.826 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /sys 2024-09-16T08:37:36.826 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /proc 2024-09-16T08:37:36.826 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /mnt 2024-09-16T08:37:36.826 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /var/tmp 2024-09-16T08:37:36.826 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /home 2024-09-16T08:37:36.826 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /root 2024-09-16T08:37:36.826 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /tmp 2024-09-16T08:37:36.826 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:37:37.865 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-radosgw-2:19.3.0-4917.g8293d73f.el9.x86_64 33/35 2024-09-16T08:37:37.890 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4917.g8293d73f.el9.x86_64 33/35 2024-09-16T08:37:37.890 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:37:37.890 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T08:37:37.890 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T08:37:37.890 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T08:37:37.890 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:37:37.891 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T08:37:37.967 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T08:37:37.967 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T08:37:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T08:37:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 1/35 2024-09-16T08:37:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 2/35 2024-09-16T08:37:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-radosgw-2:19.3.0-4917.g8293d73f.el9.x86_64 3/35 2024-09-16T08:37:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-selinux-2:19.3.0-4917.g8293d73f.el9.x86_64 4/35 2024-09-16T08:37:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 5/35 2024-09-16T08:37:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libradosstriper1-2:19.3.0-4917.g8293d73f.el9.x86_6 6/35 2024-09-16T08:37:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librgw2-2:19.3.0-4917.g8293d73f.el9.x86_64 7/35 2024-09-16T08:37:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4917.g8293d73f.el9. 8/35 2024-09-16T08:37:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-common-2:19.3.0-4917.g8293d73f.el9.x8 9/35 2024-09-16T08:37:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cephfs-2:19.3.0-4917.g8293d73f.el9.x86_64 10/35 2024-09-16T08:37:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rados-2:19.3.0-4917.g8293d73f.el9.x86_64 11/35 2024-09-16T08:37:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rbd-2:19.3.0-4917.g8293d73f.el9.x86_64 12/35 2024-09-16T08:37:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rgw-2:19.3.0-4917.g8293d73f.el9.x86_64 13/35 2024-09-16T08:37:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-16T08:37:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-16T08:37:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-16T08:37:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-16T08:37:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-16T08:37:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-16T08:37:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T08:37:39.079 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-16T08:37:39.079 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-16T08:37:39.079 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-16T08:37:39.079 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-16T08:37:39.079 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-16T08:37:39.079 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-16T08:37:39.079 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-16T08:37:39.080 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-16T08:37:39.080 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-16T08:37:39.080 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-16T08:37:39.080 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-16T08:37:39.081 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 32/35 2024-09-16T08:37:39.081 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-16T08:37:39.081 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 34/35 2024-09-16T08:37:39.900 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T08:37:39.900 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:37:39.900 INFO:teuthology.orchestra.run.smithi160.stdout:Upgraded: 2024-09-16T08:37:39.900 INFO:teuthology.orchestra.run.smithi160.stdout: librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:39.901 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:39.901 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:37:39.901 INFO:teuthology.orchestra.run.smithi160.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T08:37:39.901 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:39.901 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:39.901 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:39.901 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:39.902 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T08:37:39.902 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T08:37:39.902 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T08:37:39.902 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:39.902 INFO:teuthology.orchestra.run.smithi160.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-16T08:37:39.902 INFO:teuthology.orchestra.run.smithi160.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T08:37:39.902 INFO:teuthology.orchestra.run.smithi160.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T08:37:39.902 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T08:37:39.902 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:39.903 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T08:37:39.903 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:39.903 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T08:37:39.903 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T08:37:39.903 INFO:teuthology.orchestra.run.smithi160.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T08:37:39.903 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:39.903 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:39.904 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:39.904 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T08:37:39.904 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:39.904 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:39.904 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:39.904 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T08:37:39.904 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T08:37:39.904 INFO:teuthology.orchestra.run.smithi160.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T08:37:39.904 INFO:teuthology.orchestra.run.smithi160.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T08:37:39.905 INFO:teuthology.orchestra.run.smithi160.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T08:37:39.905 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:37:39.905 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:37:39.979 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4917.g8293d73f.el9.x86_64 32/35 2024-09-16T08:37:39.979 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2024-09-16T08:37:39.979 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2024-09-16T08:37:39.979 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2024-09-16T08:37:39.979 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2024-09-16T08:37:39.979 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2024-09-16T08:37:39.979 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /root 2024-09-16T08:37:39.980 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2024-09-16T08:37:39.980 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:37:40.276 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-test 2024-09-16T08:37:40.848 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:00:27 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:37:40.981 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:37:40.981 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:37:40.981 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-09-16T08:37:40.981 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:37:40.981 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:37:40.981 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 50 M 2024-09-16T08:37:40.981 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-09-16T08:37:40.982 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 161 k 2024-09-16T08:37:40.982 INFO:teuthology.orchestra.run.smithi160.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T08:37:40.982 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T08:37:40.982 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:37:40.982 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:37:40.982 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:37:40.982 INFO:teuthology.orchestra.run.smithi160.stdout:Install 4 Packages 2024-09-16T08:37:40.982 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:37:40.983 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 51 M 2024-09-16T08:37:40.983 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 217 M 2024-09-16T08:37:40.983 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:37:41.013 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:19.3.0-4917.g8293d73f.el9.x86_64 33/35 2024-09-16T08:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4917.g8293d73f.el9.x86_64 33/35 2024-09-16T08:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T08:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T08:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T08:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:37:41.042 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T08:37:41.124 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T08:37:41.125 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T08:37:41.321 INFO:teuthology.orchestra.run.smithi160.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-16T08:37:41.346 INFO:teuthology.orchestra.run.smithi160.stdout:(2/4): libcephsqlite-19.3.0-4917.g8293d73f.el9. 663 kB/s | 161 kB 00:00 2024-09-16T08:37:41.371 INFO:teuthology.orchestra.run.smithi160.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-16T08:37:42.281 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T08:37:42.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 1/35 2024-09-16T08:37:42.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 2/35 2024-09-16T08:37:42.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:19.3.0-4917.g8293d73f.el9.x86_64 3/35 2024-09-16T08:37:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:19.3.0-4917.g8293d73f.el9.x86_64 4/35 2024-09-16T08:37:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 5/35 2024-09-16T08:37:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:19.3.0-4917.g8293d73f.el9.x86_6 6/35 2024-09-16T08:37:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:19.3.0-4917.g8293d73f.el9.x86_64 7/35 2024-09-16T08:37:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4917.g8293d73f.el9. 8/35 2024-09-16T08:37:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:19.3.0-4917.g8293d73f.el9.x8 9/35 2024-09-16T08:37:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:19.3.0-4917.g8293d73f.el9.x86_64 10/35 2024-09-16T08:37:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:19.3.0-4917.g8293d73f.el9.x86_64 11/35 2024-09-16T08:37:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:19.3.0-4917.g8293d73f.el9.x86_64 12/35 2024-09-16T08:37:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:19.3.0-4917.g8293d73f.el9.x86_64 13/35 2024-09-16T08:37:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-16T08:37:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-16T08:37:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-16T08:37:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-16T08:37:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-16T08:37:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-16T08:37:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T08:37:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-16T08:37:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-16T08:37:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-16T08:37:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-16T08:37:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-16T08:37:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-16T08:37:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-16T08:37:42.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-16T08:37:42.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-16T08:37:42.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-16T08:37:42.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-16T08:37:42.285 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 32/35 2024-09-16T08:37:42.285 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-16T08:37:42.285 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 34/35 2024-09-16T08:37:42.671 INFO:teuthology.orchestra.run.smithi160.stdout:(4/4): ceph-test-19.3.0-4917.g8293d73f.el9.x86_ 32 MB/s | 50 MB 00:01 2024-09-16T08:37:42.674 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:37:42.674 INFO:teuthology.orchestra.run.smithi160.stdout:Total 30 MB/s | 51 MB 00:01 2024-09-16T08:37:42.701 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:37:42.722 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:37:42.722 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:37:43.125 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:37:43.125 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:37:43.184 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T08:37:43.184 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:37:43.184 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2024-09-16T08:37:43.184 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:43.184 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:43.184 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:37:43.184 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T08:37:43.184 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:43.184 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:43.184 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:43.185 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:43.185 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T08:37:43.185 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T08:37:43.185 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T08:37:43.185 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:43.185 INFO:teuthology.orchestra.run.smithi169.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-16T08:37:43.185 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T08:37:43.185 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T08:37:43.185 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T08:37:43.185 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:43.185 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T08:37:43.185 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:43.185 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T08:37:43.186 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T08:37:43.186 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T08:37:43.186 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:43.186 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:43.186 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:43.186 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T08:37:43.186 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:43.186 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:43.186 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:43.186 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T08:37:43.186 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T08:37:43.186 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T08:37:43.186 INFO:teuthology.orchestra.run.smithi169.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T08:37:43.187 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T08:37:43.187 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:37:43.187 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:37:43.583 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-test 2024-09-16T08:37:43.707 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:37:43.811 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T08:37:43.860 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T08:37:43.890 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephsqlite-2:19.3.0-4917.g8293d73f.el9.x86_64 3/4 2024-09-16T08:37:44.157 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:31 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:37:44.290 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:37:44.291 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:37:44.291 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T08:37:44.291 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:37:44.291 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:37:44.291 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 50 M 2024-09-16T08:37:44.291 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T08:37:44.291 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 161 k 2024-09-16T08:37:44.291 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T08:37:44.291 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T08:37:44.291 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:37:44.291 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:37:44.291 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:37:44.292 INFO:teuthology.orchestra.run.smithi169.stdout:Install 4 Packages 2024-09-16T08:37:44.292 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:37:44.292 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 51 M 2024-09-16T08:37:44.292 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 217 M 2024-09-16T08:37:44.292 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:37:44.553 INFO:teuthology.orchestra.run.smithi169.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-16T08:37:44.595 INFO:teuthology.orchestra.run.smithi169.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-16T08:37:44.653 INFO:teuthology.orchestra.run.smithi169.stdout:(3/4): libcephsqlite-19.3.0-4917.g8293d73f.el9. 715 kB/s | 161 kB 00:00 2024-09-16T08:37:46.404 INFO:teuthology.orchestra.run.smithi169.stdout:(4/4): ceph-test-19.3.0-4917.g8293d73f.el9.x86_ 25 MB/s | 50 MB 00:01 2024-09-16T08:37:46.406 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:37:46.406 INFO:teuthology.orchestra.run.smithi169.stdout:Total 24 MB/s | 51 MB 00:02 2024-09-16T08:37:46.434 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:37:46.454 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:37:46.455 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:37:46.854 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:37:46.854 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:37:47.437 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:37:47.523 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T08:37:47.588 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T08:37:47.614 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:19.3.0-4917.g8293d73f.el9.x86_64 3/4 2024-09-16T08:37:48.665 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4917.g8293d73f.el9.x86_64 3/4 2024-09-16T08:37:48.754 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-test-2:19.3.0-4917.g8293d73f.el9.x86_64 4/4 2024-09-16T08:37:51.187 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-test-2:19.3.0-4917.g8293d73f.el9.x86_64 4/4 2024-09-16T08:37:51.188 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-test-2:19.3.0-4917.g8293d73f.el9.x86_64 1/4 2024-09-16T08:37:51.188 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephsqlite-2:19.3.0-4917.g8293d73f.el9.x86_64 2/4 2024-09-16T08:37:51.188 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T08:37:51.470 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T08:37:51.470 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:37:51.470 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:37:51.470 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:51.470 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:51.470 INFO:teuthology.orchestra.run.smithi160.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T08:37:51.470 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T08:37:51.470 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:37:51.470 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:37:51.742 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph 2024-09-16T08:37:52.329 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:00:39 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:37:52.412 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4917.g8293d73f.el9.x86_64 3/4 2024-09-16T08:37:52.472 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:37:52.475 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:37:52.475 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-09-16T08:37:52.475 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:37:52.475 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:37:52.475 INFO:teuthology.orchestra.run.smithi160.stdout: ceph x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 6.6 k 2024-09-16T08:37:52.475 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-09-16T08:37:52.475 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 2.4 M 2024-09-16T08:37:52.475 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 1.5 M 2024-09-16T08:37:52.475 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 291 k 2024-09-16T08:37:52.475 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 4.8 M 2024-09-16T08:37:52.476 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 17 M 2024-09-16T08:37:52.476 INFO:teuthology.orchestra.run.smithi160.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T08:37:52.476 INFO:teuthology.orchestra.run.smithi160.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T08:37:52.476 INFO:teuthology.orchestra.run.smithi160.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T08:37:52.476 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T08:37:52.476 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T08:37:52.476 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T08:37:52.476 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T08:37:52.476 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T08:37:52.476 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T08:37:52.476 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T08:37:52.476 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-collections 2024-09-16T08:37:52.477 INFO:teuthology.orchestra.run.smithi160.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-16T08:37:52.477 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T08:37:52.477 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T08:37:52.477 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T08:37:52.477 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T08:37:52.477 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T08:37:52.477 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T08:37:52.477 INFO:teuthology.orchestra.run.smithi160.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T08:37:52.477 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T08:37:52.477 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T08:37:52.477 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T08:37:52.477 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T08:37:52.478 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T08:37:52.478 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T08:37:52.478 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T08:37:52.478 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T08:37:52.478 INFO:teuthology.orchestra.run.smithi160.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T08:37:52.478 INFO:teuthology.orchestra.run.smithi160.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T08:37:52.478 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T08:37:52.478 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T08:37:52.478 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T08:37:52.478 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:37:52.478 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:37:52.478 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:37:52.478 INFO:teuthology.orchestra.run.smithi160.stdout:Install 37 Packages 2024-09-16T08:37:52.478 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:37:52.479 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 31 M 2024-09-16T08:37:52.479 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 109 M 2024-09-16T08:37:52.479 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:37:52.497 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:19.3.0-4917.g8293d73f.el9.x86_64 4/4 2024-09-16T08:37:53.281 INFO:teuthology.orchestra.run.smithi160.stdout:(1/37): ceph-19.3.0-4917.g8293d73f.el9.x86_64.r 50 kB/s | 6.6 kB 00:00 2024-09-16T08:37:53.497 INFO:teuthology.orchestra.run.smithi160.stdout:(2/37): ceph-mds-19.3.0-4917.g8293d73f.el9.x86_ 6.8 MB/s | 2.4 MB 00:00 2024-09-16T08:37:53.556 INFO:teuthology.orchestra.run.smithi160.stdout:(3/37): ceph-mgr-19.3.0-4917.g8293d73f.el9.x86_ 3.7 MB/s | 1.5 MB 00:00 2024-09-16T08:37:53.698 INFO:teuthology.orchestra.run.smithi160.stdout:(4/37): ceph-mon-19.3.0-4917.g8293d73f.el9.x86_ 11 MB/s | 4.8 MB 00:00 2024-09-16T08:37:53.723 INFO:teuthology.orchestra.run.smithi160.stdout:(5/37): ceph-mgr-modules-core-19.3.0-4917.g8293 1.7 MB/s | 291 kB 00:00 2024-09-16T08:37:53.907 INFO:teuthology.orchestra.run.smithi160.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-16T08:37:54.074 INFO:teuthology.orchestra.run.smithi160.stdout:(7/37): ceph-osd-19.3.0-4917.g8293d73f.el9.x86_ 30 MB/s | 17 MB 00:00 2024-09-16T08:37:54.099 INFO:teuthology.orchestra.run.smithi160.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 555 kB/s | 106 kB 00:00 2024-09-16T08:37:54.149 INFO:teuthology.orchestra.run.smithi160.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-16T08:37:54.199 INFO:teuthology.orchestra.run.smithi160.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 2.6 MB/s | 1.2 MB 00:00 2024-09-16T08:37:54.225 INFO:teuthology.orchestra.run.smithi160.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 2.8 MB/s | 215 kB 00:00 2024-09-16T08:37:54.308 INFO:teuthology.orchestra.run.smithi160.stdout:(12/37): python3-pycparser-2.20-6.el9.noarch.rp 577 kB/s | 135 kB 00:00 2024-09-16T08:37:54.334 INFO:teuthology.orchestra.run.smithi160.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-09-16T08:37:54.359 INFO:teuthology.orchestra.run.smithi160.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-16T08:37:54.384 INFO:teuthology.orchestra.run.smithi160.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-16T08:37:54.409 INFO:teuthology.orchestra.run.smithi160.stdout:(16/37): python3-markupsafe-1.1.1-12.el9.x86_64 345 kB/s | 35 kB 00:00 2024-09-16T08:37:54.435 INFO:teuthology.orchestra.run.smithi160.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 674 kB/s | 51 kB 00:00 2024-09-16T08:37:54.460 INFO:teuthology.orchestra.run.smithi160.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 551 kB/s | 42 kB 00:00 2024-09-16T08:37:54.485 INFO:teuthology.orchestra.run.smithi160.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 294 kB/s | 22 kB 00:00 2024-09-16T08:37:54.536 INFO:teuthology.orchestra.run.smithi160.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-09-16T08:37:54.561 INFO:teuthology.orchestra.run.smithi160.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 432 kB/s | 43 kB 00:00 2024-09-16T08:37:54.586 INFO:teuthology.orchestra.run.smithi160.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 428 kB/s | 11 kB 00:00 2024-09-16T08:37:54.611 INFO:teuthology.orchestra.run.smithi160.stdout:(23/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-16T08:37:54.637 INFO:teuthology.orchestra.run.smithi160.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-16T08:37:54.662 INFO:teuthology.orchestra.run.smithi160.stdout:(25/37): python3-jaraco-functools-3.5.0-2.el9.n 786 kB/s | 19 kB 00:00 2024-09-16T08:37:54.687 INFO:teuthology.orchestra.run.smithi160.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-16T08:37:54.712 INFO:teuthology.orchestra.run.smithi160.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 390 kB/s | 20 kB 00:00 2024-09-16T08:37:54.740 INFO:teuthology.orchestra.run.smithi160.stdout:(28/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-09-16T08:37:54.765 INFO:teuthology.orchestra.run.smithi160.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 595 kB/s | 46 kB 00:00 2024-09-16T08:37:54.790 INFO:teuthology.orchestra.run.smithi160.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-16T08:37:54.816 INFO:teuthology.orchestra.run.smithi160.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-16T08:37:54.841 INFO:teuthology.orchestra.run.smithi160.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-16T08:37:54.866 INFO:teuthology.orchestra.run.smithi160.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-09-16T08:37:54.891 INFO:teuthology.orchestra.run.smithi160.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-16T08:37:54.917 INFO:teuthology.orchestra.run.smithi160.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 801 kB/s | 20 kB 00:00 2024-09-16T08:37:54.950 INFO:teuthology.orchestra.run.smithi160.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-16T08:37:54.982 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-test-2:19.3.0-4917.g8293d73f.el9.x86_64 4/4 2024-09-16T08:37:54.982 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:19.3.0-4917.g8293d73f.el9.x86_64 1/4 2024-09-16T08:37:54.982 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:19.3.0-4917.g8293d73f.el9.x86_64 2/4 2024-09-16T08:37:54.982 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T08:37:54.984 INFO:teuthology.orchestra.run.smithi160.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-09-16T08:37:54.987 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:37:54.988 INFO:teuthology.orchestra.run.smithi160.stdout:Total 12 MB/s | 31 MB 00:02 2024-09-16T08:37:55.133 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:37:55.173 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:37:55.173 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:37:55.259 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T08:37:55.259 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:37:55.259 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:37:55.259 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:55.259 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:37:55.259 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T08:37:55.259 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T08:37:55.259 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:37:55.259 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:37:55.548 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph 2024-09-16T08:37:55.627 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:37:55.627 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:37:56.151 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:43 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:37:56.293 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:37:56.295 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:37:56.295 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T08:37:56.296 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:37:56.296 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:37:56.296 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 6.6 k 2024-09-16T08:37:56.296 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T08:37:56.296 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 2.4 M 2024-09-16T08:37:56.296 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 1.5 M 2024-09-16T08:37:56.296 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 291 k 2024-09-16T08:37:56.296 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 4.8 M 2024-09-16T08:37:56.296 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 17 M 2024-09-16T08:37:56.296 INFO:teuthology.orchestra.run.smithi169.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T08:37:56.296 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T08:37:56.297 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T08:37:56.297 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T08:37:56.297 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T08:37:56.297 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T08:37:56.297 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T08:37:56.297 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T08:37:56.297 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T08:37:56.297 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T08:37:56.297 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections 2024-09-16T08:37:56.297 INFO:teuthology.orchestra.run.smithi169.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-16T08:37:56.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T08:37:56.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T08:37:56.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T08:37:56.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T08:37:56.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T08:37:56.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T08:37:56.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T08:37:56.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T08:37:56.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T08:37:56.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T08:37:56.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T08:37:56.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T08:37:56.299 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T08:37:56.299 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T08:37:56.299 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T08:37:56.299 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T08:37:56.299 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T08:37:56.299 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T08:37:56.299 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T08:37:56.299 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T08:37:56.299 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:37:56.299 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:37:56.299 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:37:56.299 INFO:teuthology.orchestra.run.smithi169.stdout:Install 37 Packages 2024-09-16T08:37:56.300 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:37:56.300 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 31 M 2024-09-16T08:37:56.300 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 109 M 2024-09-16T08:37:56.300 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:37:56.592 INFO:teuthology.orchestra.run.smithi169.stdout:(1/37): ceph-19.3.0-4917.g8293d73f.el9.x86_64.r 46 kB/s | 6.6 kB 00:00 2024-09-16T08:37:56.750 INFO:teuthology.orchestra.run.smithi169.stdout:(2/37): ceph-mgr-19.3.0-4917.g8293d73f.el9.x86_ 5.1 MB/s | 1.5 MB 00:00 2024-09-16T08:37:56.801 INFO:teuthology.orchestra.run.smithi169.stdout:(3/37): ceph-mds-19.3.0-4917.g8293d73f.el9.x86_ 6.8 MB/s | 2.4 MB 00:00 2024-09-16T08:37:56.901 INFO:teuthology.orchestra.run.smithi169.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4917.g8293 2.8 MB/s | 291 kB 00:00 2024-09-16T08:37:56.927 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:37:57.003 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-16T08:37:57.026 INFO:teuthology.orchestra.run.smithi169.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-09-16T08:37:57.036 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-16T08:37:57.064 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-16T08:37:57.135 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-16T08:37:57.185 INFO:teuthology.orchestra.run.smithi169.stdout:(6/37): ceph-mon-19.3.0-4917.g8293d73f.el9.x86_ 8.1 MB/s | 4.8 MB 00:00 2024-09-16T08:37:57.213 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-16T08:37:57.254 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-16T08:37:57.352 INFO:teuthology.orchestra.run.smithi169.stdout:(7/37): ceph-osd-19.3.0-4917.g8293d73f.el9.x86_ 29 MB/s | 17 MB 00:00 2024-09-16T08:37:57.363 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-16T08:37:57.402 INFO:teuthology.orchestra.run.smithi169.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 3.3 MB/s | 1.2 MB 00:00 2024-09-16T08:37:57.444 INFO:teuthology.orchestra.run.smithi169.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.9 MB/s | 125 kB 00:00 2024-09-16T08:37:57.449 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-16T08:37:57.469 INFO:teuthology.orchestra.run.smithi169.stdout:(10/37): python3-ply-3.11-14.el9.noarch.rpm 375 kB/s | 106 kB 00:00 2024-09-16T08:37:57.490 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-16T08:37:57.528 INFO:teuthology.orchestra.run.smithi169.stdout:(11/37): python3-pycparser-2.20-6.el9.noarch.rp 769 kB/s | 135 kB 00:00 2024-09-16T08:37:57.533 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-16T08:37:57.553 INFO:teuthology.orchestra.run.smithi169.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.9 MB/s | 215 kB 00:00 2024-09-16T08:37:57.563 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-16T08:37:57.598 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-16T08:37:57.603 INFO:teuthology.orchestra.run.smithi169.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-09-16T08:37:57.631 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-16T08:37:57.654 INFO:teuthology.orchestra.run.smithi169.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 347 kB/s | 35 kB 00:00 2024-09-16T08:37:57.672 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-16T08:37:57.679 INFO:teuthology.orchestra.run.smithi169.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.0 MB/s | 77 kB 00:00 2024-09-16T08:37:57.713 INFO:teuthology.orchestra.run.smithi169.stdout:(16/37): python3-mako-1.1.4-6.el9.noarch.rpm 933 kB/s | 172 kB 00:00 2024-09-16T08:37:57.715 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-16T08:37:57.738 INFO:teuthology.orchestra.run.smithi169.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 607 kB/s | 51 kB 00:00 2024-09-16T08:37:57.758 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-16T08:37:57.763 INFO:teuthology.orchestra.run.smithi169.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 497 kB/s | 42 kB 00:00 2024-09-16T08:37:57.789 INFO:teuthology.orchestra.run.smithi169.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 295 kB/s | 22 kB 00:00 2024-09-16T08:37:57.815 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-16T08:37:57.839 INFO:teuthology.orchestra.run.smithi169.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-09-16T08:37:57.846 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-16T08:37:57.864 INFO:teuthology.orchestra.run.smithi169.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 433 kB/s | 43 kB 00:00 2024-09-16T08:37:57.889 INFO:teuthology.orchestra.run.smithi169.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 430 kB/s | 11 kB 00:00 2024-09-16T08:37:57.915 INFO:teuthology.orchestra.run.smithi169.stdout:(23/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-16T08:37:57.919 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-16T08:37:57.940 INFO:teuthology.orchestra.run.smithi169.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 351 kB/s | 18 kB 00:00 2024-09-16T08:37:57.965 INFO:teuthology.orchestra.run.smithi169.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 459 kB/s | 23 kB 00:00 2024-09-16T08:37:57.991 INFO:teuthology.orchestra.run.smithi169.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-09-16T08:37:58.016 INFO:teuthology.orchestra.run.smithi169.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-16T08:37:58.041 INFO:teuthology.orchestra.run.smithi169.stdout:(28/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-09-16T08:37:58.058 INFO:teuthology.orchestra.run.smithi169.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 692 kB/s | 46 kB 00:00 2024-09-16T08:37:58.066 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-16T08:37:58.083 INFO:teuthology.orchestra.run.smithi169.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.1 MB/s | 79 kB 00:00 2024-09-16T08:37:58.100 INFO:teuthology.orchestra.run.smithi169.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 393 kB/s | 16 kB 00:00 2024-09-16T08:37:58.125 INFO:teuthology.orchestra.run.smithi169.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-09-16T08:37:58.143 INFO:teuthology.orchestra.run.smithi169.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 855 kB/s | 36 kB 00:00 2024-09-16T08:37:58.156 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-16T08:37:58.168 INFO:teuthology.orchestra.run.smithi169.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-16T08:37:58.193 INFO:teuthology.orchestra.run.smithi169.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 811 kB/s | 20 kB 00:00 2024-09-16T08:37:58.208 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-16T08:37:58.228 INFO:teuthology.orchestra.run.smithi169.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-16T08:37:58.246 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-16T08:37:58.260 INFO:teuthology.orchestra.run.smithi169.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-09-16T08:37:58.265 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:37:58.265 INFO:teuthology.orchestra.run.smithi169.stdout:Total 16 MB/s | 31 MB 00:01 2024-09-16T08:37:58.312 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-16T08:37:58.376 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-16T08:37:58.415 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:37:58.456 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:37:58.457 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:37:58.562 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-16T08:37:58.608 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-16T08:37:58.679 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-16T08:37:58.768 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-16T08:37:58.814 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-16T08:37:58.921 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:37:58.921 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:37:58.943 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-16T08:37:59.107 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4917.g8293d73f.el9. 32/37 2024-09-16T08:37:59.135 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 33/37 2024-09-16T08:37:59.162 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 33/37 2024-09-16T08:37:59.163 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:37:59.163 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T08:37:59.163 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T08:37:59.163 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T08:37:59.163 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:00.230 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:38:00.287 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-16T08:38:00.320 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-16T08:38:00.348 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-16T08:38:00.419 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-16T08:38:00.472 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-16T08:38:00.513 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-16T08:38:00.613 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-16T08:38:00.627 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-osd-2:19.3.0-4917.g8293d73f.el9.x86_64 34/37 2024-09-16T08:38:00.655 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-osd-2:19.3.0-4917.g8293d73f.el9.x86_64 34/37 2024-09-16T08:38:00.655 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:38:00.655 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T08:38:00.655 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T08:38:00.655 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T08:38:00.655 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:00.692 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-16T08:38:00.741 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-16T08:38:00.785 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-16T08:38:00.815 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-16T08:38:00.849 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-16T08:38:00.882 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-16T08:38:00.923 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-16T08:38:00.967 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-16T08:38:01.017 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-16T08:38:01.066 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-16T08:38:01.103 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mon-2:19.3.0-4917.g8293d73f.el9.x86_64 35/37 2024-09-16T08:38:01.106 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-16T08:38:01.127 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mon-2:19.3.0-4917.g8293d73f.el9.x86_64 35/37 2024-09-16T08:38:01.127 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:38:01.127 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T08:38:01.127 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T08:38:01.127 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T08:38:01.127 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:01.179 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-16T08:38:01.327 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-16T08:38:01.354 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mds-2:19.3.0-4917.g8293d73f.el9.x86_64 36/37 2024-09-16T08:38:01.378 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mds-2:19.3.0-4917.g8293d73f.el9.x86_64 36/37 2024-09-16T08:38:01.378 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:38:01.378 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T08:38:01.378 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T08:38:01.378 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T08:38:01.378 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:01.415 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-16T08:38:01.442 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-2:19.3.0-4917.g8293d73f.el9.x86_64 37/37 2024-09-16T08:38:01.467 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-16T08:38:01.509 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-16T08:38:01.563 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-16T08:38:01.619 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-16T08:38:01.815 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-16T08:38:01.860 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-16T08:38:01.931 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-16T08:38:02.028 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-16T08:38:02.076 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-16T08:38:02.205 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-16T08:38:02.374 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4917.g8293d73f.el9. 32/37 2024-09-16T08:38:02.404 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 33/37 2024-09-16T08:38:02.430 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 33/37 2024-09-16T08:38:02.430 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:38:02.430 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T08:38:02.430 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T08:38:02.430 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T08:38:02.430 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:03.386 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-2:19.3.0-4917.g8293d73f.el9.x86_64 37/37 2024-09-16T08:38:03.386 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-2:19.3.0-4917.g8293d73f.el9.x86_64 1/37 2024-09-16T08:38:03.386 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mds-2:19.3.0-4917.g8293d73f.el9.x86_64 2/37 2024-09-16T08:38:03.386 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 3/37 2024-09-16T08:38:03.386 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mon-2:19.3.0-4917.g8293d73f.el9.x86_64 4/37 2024-09-16T08:38:03.386 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-osd-2:19.3.0-4917.g8293d73f.el9.x86_64 5/37 2024-09-16T08:38:03.386 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4917.g8293d73f.el9. 6/37 2024-09-16T08:38:03.386 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-16T08:38:03.386 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-16T08:38:03.387 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-16T08:38:03.387 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-16T08:38:03.387 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-16T08:38:03.387 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-16T08:38:03.387 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-16T08:38:03.387 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-16T08:38:03.387 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-16T08:38:03.387 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-16T08:38:03.387 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-16T08:38:03.387 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-16T08:38:03.387 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-16T08:38:03.387 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-16T08:38:03.388 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-16T08:38:03.388 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-16T08:38:03.388 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-16T08:38:03.388 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-16T08:38:03.388 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-16T08:38:03.388 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-16T08:38:03.388 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-16T08:38:03.388 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-16T08:38:03.388 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-16T08:38:03.388 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-16T08:38:03.388 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-16T08:38:03.388 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-16T08:38:03.389 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-16T08:38:03.389 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-16T08:38:03.389 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-16T08:38:03.389 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-16T08:38:03.912 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:19.3.0-4917.g8293d73f.el9.x86_64 34/37 2024-09-16T08:38:03.940 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:19.3.0-4917.g8293d73f.el9.x86_64 34/37 2024-09-16T08:38:03.940 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:38:03.940 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T08:38:03.940 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T08:38:03.940 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T08:38:03.940 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:04.314 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-16T08:38:04.315 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:04.315 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:38:04.315 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:38:04.315 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:38:04.315 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:38:04.315 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:04.315 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:38:04.315 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:38:04.315 INFO:teuthology.orchestra.run.smithi160.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T08:38:04.315 INFO:teuthology.orchestra.run.smithi160.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T08:38:04.315 INFO:teuthology.orchestra.run.smithi160.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T08:38:04.315 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T08:38:04.315 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T08:38:04.316 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T08:38:04.316 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T08:38:04.316 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T08:38:04.316 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T08:38:04.316 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T08:38:04.316 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T08:38:04.316 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T08:38:04.316 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T08:38:04.316 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T08:38:04.316 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T08:38:04.316 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T08:38:04.316 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T08:38:04.317 INFO:teuthology.orchestra.run.smithi160.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T08:38:04.317 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T08:38:04.317 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T08:38:04.317 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T08:38:04.317 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T08:38:04.317 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T08:38:04.317 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T08:38:04.317 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T08:38:04.317 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T08:38:04.317 INFO:teuthology.orchestra.run.smithi160.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T08:38:04.317 INFO:teuthology.orchestra.run.smithi160.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T08:38:04.317 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T08:38:04.317 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T08:38:04.318 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T08:38:04.318 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:04.318 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:04.397 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:19.3.0-4917.g8293d73f.el9.x86_64 35/37 2024-09-16T08:38:04.420 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:19.3.0-4917.g8293d73f.el9.x86_64 35/37 2024-09-16T08:38:04.421 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:38:04.421 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T08:38:04.421 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T08:38:04.421 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T08:38:04.421 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:04.655 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:19.3.0-4917.g8293d73f.el9.x86_64 36/37 2024-09-16T08:38:04.679 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:19.3.0-4917.g8293d73f.el9.x86_64 36/37 2024-09-16T08:38:04.680 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:38:04.680 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T08:38:04.680 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T08:38:04.680 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T08:38:04.680 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:04.693 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-base 2024-09-16T08:38:04.745 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:19.3.0-4917.g8293d73f.el9.x86_64 37/37 2024-09-16T08:38:05.264 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:00:52 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:05.354 INFO:teuthology.orchestra.run.smithi160.stdout:Package ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:38:05.397 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:05.398 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-09-16T08:38:05.398 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:05.474 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install cephadm 2024-09-16T08:38:06.035 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:00:53 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:06.166 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:06.167 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:06.167 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-09-16T08:38:06.167 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:06.167 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:38:06.167 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 784 k 2024-09-16T08:38:06.167 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:06.167 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:38:06.168 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:06.168 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2024-09-16T08:38:06.168 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:06.168 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 784 k 2024-09-16T08:38:06.168 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 791 k 2024-09-16T08:38:06.168 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:38:06.512 INFO:teuthology.orchestra.run.smithi160.stdout:cephadm-19.3.0-4917.g8293d73f.el9.noarch.rpm 2.2 MB/s | 784 kB 00:00 2024-09-16T08:38:06.512 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:06.513 INFO:teuthology.orchestra.run.smithi160.stdout:Total 2.2 MB/s | 784 kB 00:00 2024-09-16T08:38:06.513 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:38:06.518 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:38:06.518 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:38:06.524 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:38:06.524 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:38:06.568 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-2:19.3.0-4917.g8293d73f.el9.x86_64 37/37 2024-09-16T08:38:06.569 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:19.3.0-4917.g8293d73f.el9.x86_64 1/37 2024-09-16T08:38:06.569 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:19.3.0-4917.g8293d73f.el9.x86_64 2/37 2024-09-16T08:38:06.569 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 3/37 2024-09-16T08:38:06.569 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:19.3.0-4917.g8293d73f.el9.x86_64 4/37 2024-09-16T08:38:06.569 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:19.3.0-4917.g8293d73f.el9.x86_64 5/37 2024-09-16T08:38:06.569 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4917.g8293d73f.el9. 6/37 2024-09-16T08:38:06.569 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-16T08:38:06.569 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-16T08:38:06.569 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-16T08:38:06.569 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-16T08:38:06.569 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-16T08:38:06.570 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-16T08:38:06.570 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-16T08:38:06.570 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-16T08:38:06.570 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-16T08:38:06.570 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-16T08:38:06.570 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-16T08:38:06.570 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-16T08:38:06.570 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-16T08:38:06.570 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-16T08:38:06.570 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-16T08:38:06.570 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-16T08:38:06.570 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-16T08:38:06.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-16T08:38:06.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-16T08:38:06.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-16T08:38:06.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-16T08:38:06.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-16T08:38:06.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-16T08:38:06.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-16T08:38:06.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-16T08:38:06.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-16T08:38:06.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-16T08:38:06.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-16T08:38:06.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-16T08:38:06.572 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-16T08:38:06.599 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:38:07.024 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T08:38:07.141 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T08:38:07.515 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T08:38:07.583 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-16T08:38:07.583 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:07.583 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:38:07.583 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:38:07.583 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:38:07.583 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:38:07.583 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:07.584 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:38:07.584 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:38:07.584 INFO:teuthology.orchestra.run.smithi169.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T08:38:07.584 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T08:38:07.584 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T08:38:07.584 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T08:38:07.584 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T08:38:07.584 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T08:38:07.584 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T08:38:07.584 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T08:38:07.584 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T08:38:07.584 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T08:38:07.585 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T08:38:07.585 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T08:38:07.585 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T08:38:07.585 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T08:38:07.585 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T08:38:07.585 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T08:38:07.585 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T08:38:07.585 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T08:38:07.585 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T08:38:07.585 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T08:38:07.585 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T08:38:07.585 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T08:38:07.585 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T08:38:07.586 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T08:38:07.586 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T08:38:07.586 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T08:38:07.586 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T08:38:07.586 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T08:38:07.586 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T08:38:07.586 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T08:38:07.586 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T08:38:07.586 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:07.586 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:38:07.774 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T08:38:07.775 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:07.775 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:38:07.775 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:07.775 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:07.775 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:07.982 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T08:38:07.997 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-base 2024-09-16T08:38:08.562 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:00:55 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:08.567 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:55 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:08.659 INFO:teuthology.orchestra.run.smithi169.stdout:Package ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:38:08.697 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:08.698 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:08.698 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repo Size 2024-09-16T08:38:08.698 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:08.698 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:38:08.698 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 145 k 2024-09-16T08:38:08.699 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:08.699 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:38:08.699 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:08.699 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2024-09-16T08:38:08.699 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:08.699 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 145 k 2024-09-16T08:38:08.699 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 432 k 2024-09-16T08:38:08.699 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:38:08.701 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:38:08.703 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T08:38:08.703 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:38:08.779 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install cephadm 2024-09-16T08:38:08.910 INFO:teuthology.orchestra.run.smithi160.stdout:ceph-immutable-object-cache-19.3.0-4917.g8293d7 689 kB/s | 145 kB 00:00 2024-09-16T08:38:08.911 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:08.911 INFO:teuthology.orchestra.run.smithi160.stdout:Total 683 kB/s | 145 kB 00:00 2024-09-16T08:38:08.911 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:38:08.919 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:38:08.919 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:38:08.977 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:38:08.978 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:38:09.141 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:38:09.228 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4917.g8293d73f. 1/1 2024-09-16T08:38:09.252 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4917.g8293d73f. 1/1 2024-09-16T08:38:09.253 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:38:09.253 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T08:38:09.253 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:09.343 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:56 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:09.476 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:38:09.477 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:09.477 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T08:38:09.477 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:09.477 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:38:09.477 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 784 k 2024-09-16T08:38:09.477 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:09.477 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:38:09.477 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:09.477 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-09-16T08:38:09.478 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:09.478 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 784 k 2024-09-16T08:38:09.478 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 791 k 2024-09-16T08:38:09.478 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:38:09.746 INFO:teuthology.orchestra.run.smithi169.stdout:cephadm-19.3.0-4917.g8293d73f.el9.noarch.rpm 2.9 MB/s | 784 kB 00:00 2024-09-16T08:38:09.747 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:09.747 INFO:teuthology.orchestra.run.smithi169.stdout:Total 2.8 MB/s | 784 kB 00:00 2024-09-16T08:38:09.747 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:38:09.752 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:38:09.752 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:38:09.758 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:38:09.758 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:38:09.834 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:38:10.169 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4917.g8293d73f. 1/1 2024-09-16T08:38:10.169 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:10.169 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:38:10.169 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:38:10.169 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:10.169 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:10.266 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T08:38:10.367 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T08:38:10.396 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-mgr 2024-09-16T08:38:10.723 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T08:38:10.973 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:00:57 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:11.005 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T08:38:11.005 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:11.005 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:38:11.005 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:11.005 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:11.005 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:38:11.067 INFO:teuthology.orchestra.run.smithi160.stdout:Package ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:38:11.111 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:11.112 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-09-16T08:38:11.112 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:11.183 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T08:38:11.250 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T08:38:11.745 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:00:58 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:11.821 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:58 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:11.877 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:11.878 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:11.878 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-09-16T08:38:11.878 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:11.878 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:38:11.878 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 14 M 2024-09-16T08:38:11.878 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-09-16T08:38:11.878 INFO:teuthology.orchestra.run.smithi160.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T08:38:11.878 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 34 k 2024-09-16T08:38:11.878 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 16 k 2024-09-16T08:38:11.878 INFO:teuthology.orchestra.run.smithi160.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T08:38:11.878 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T08:38:11.879 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T08:38:11.879 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T08:38:11.879 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T08:38:11.879 INFO:teuthology.orchestra.run.smithi160.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T08:38:11.879 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T08:38:11.879 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T08:38:11.879 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:11.879 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:38:11.879 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:11.879 INFO:teuthology.orchestra.run.smithi160.stdout:Install 12 Packages 2024-09-16T08:38:11.879 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:11.880 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 19 M 2024-09-16T08:38:11.880 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 159 M 2024-09-16T08:38:11.880 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:38:11.954 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:38:11.955 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:11.955 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2024-09-16T08:38:11.955 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:11.955 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:38:11.955 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 145 k 2024-09-16T08:38:11.955 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:11.955 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:38:11.955 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:11.956 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-09-16T08:38:11.956 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:11.956 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 145 k 2024-09-16T08:38:11.956 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 432 k 2024-09-16T08:38:11.956 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:38:12.122 INFO:teuthology.orchestra.run.smithi160.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4917.g82 200 kB/s | 34 kB 00:00 2024-09-16T08:38:12.148 INFO:teuthology.orchestra.run.smithi160.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4917.g829 84 kB/s | 16 kB 00:00 2024-09-16T08:38:12.169 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-immutable-object-cache-19.3.0-4917.g8293d7 684 kB/s | 145 kB 00:00 2024-09-16T08:38:12.169 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:12.170 INFO:teuthology.orchestra.run.smithi169.stdout:Total 679 kB/s | 145 kB 00:00 2024-09-16T08:38:12.170 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:38:12.177 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:38:12.177 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:38:12.235 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:38:12.235 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:38:12.323 INFO:teuthology.orchestra.run.smithi160.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.5 MB/s | 269 kB 00:00 2024-09-16T08:38:12.382 INFO:teuthology.orchestra.run.smithi160.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-09-16T08:38:12.394 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:38:12.449 INFO:teuthology.orchestra.run.smithi160.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 8.1 MB/s | 548 kB 00:00 2024-09-16T08:38:12.474 INFO:teuthology.orchestra.run.smithi160.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 786 kB/s | 19 kB 00:00 2024-09-16T08:38:12.477 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4917.g8293d73f. 1/1 2024-09-16T08:38:12.502 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4917.g8293d73f. 1/1 2024-09-16T08:38:12.503 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:38:12.503 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T08:38:12.503 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:12.641 INFO:teuthology.orchestra.run.smithi160.stdout:(7/12): ceph-mgr-dashboard-19.3.0-4917.g8293d73 21 MB/s | 14 MB 00:00 2024-09-16T08:38:12.708 INFO:teuthology.orchestra.run.smithi160.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 8.7 MB/s | 2.0 MB 00:00 2024-09-16T08:38:12.733 INFO:teuthology.orchestra.run.smithi160.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.5 MB/s | 144 kB 00:00 2024-09-16T08:38:12.758 INFO:teuthology.orchestra.run.smithi160.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 616 kB/s | 31 kB 00:00 2024-09-16T08:38:12.792 INFO:teuthology.orchestra.run.smithi160.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.1 MB/s | 188 kB 00:00 2024-09-16T08:38:12.817 INFO:teuthology.orchestra.run.smithi160.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 1.7 MB/s | 863 kB 00:00 2024-09-16T08:38:12.820 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:12.820 INFO:teuthology.orchestra.run.smithi160.stdout:Total 20 MB/s | 19 MB 00:00 2024-09-16T08:38:12.868 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:38:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:38:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:38:13.164 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:38:13.164 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:38:13.413 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4917.g8293d73f. 1/1 2024-09-16T08:38:13.413 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:13.413 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:38:13.413 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:38:13.413 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:13.413 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:38:13.657 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr 2024-09-16T08:38:13.761 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:38:13.834 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-16T08:38:13.878 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-16T08:38:13.925 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-16T08:38:13.963 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-16T08:38:14.060 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-16T08:38:14.191 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-16T08:38:14.237 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:01 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:14.284 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-16T08:38:14.333 INFO:teuthology.orchestra.run.smithi169.stdout:Package ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:38:14.343 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-16T08:38:14.371 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-16T08:38:14.376 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:38:14.378 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T08:38:14.378 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:38:14.403 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4917.g8293d73f.el9 10/12 2024-09-16T08:38:14.450 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T08:38:15.018 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:02 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:15.153 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:38:15.153 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:15.154 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T08:38:15.154 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:15.154 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:38:15.154 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 14 M 2024-09-16T08:38:15.154 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T08:38:15.154 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T08:38:15.154 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 34 k 2024-09-16T08:38:15.154 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 16 k 2024-09-16T08:38:15.154 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T08:38:15.154 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T08:38:15.154 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T08:38:15.155 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T08:38:15.155 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T08:38:15.155 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T08:38:15.155 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T08:38:15.155 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T08:38:15.155 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:15.155 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:38:15.155 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:15.155 INFO:teuthology.orchestra.run.smithi169.stdout:Install 12 Packages 2024-09-16T08:38:15.155 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:15.156 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 19 M 2024-09-16T08:38:15.156 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 159 M 2024-09-16T08:38:15.156 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:38:15.864 INFO:teuthology.orchestra.run.smithi169.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4917.g82 213 kB/s | 34 kB 00:00 2024-09-16T08:38:15.890 INFO:teuthology.orchestra.run.smithi169.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4917.g829 89 kB/s | 16 kB 00:00 2024-09-16T08:38:16.023 INFO:teuthology.orchestra.run.smithi169.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.0 MB/s | 269 kB 00:00 2024-09-16T08:38:16.065 INFO:teuthology.orchestra.run.smithi169.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-09-16T08:38:16.115 INFO:teuthology.orchestra.run.smithi169.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-16T08:38:16.141 INFO:teuthology.orchestra.run.smithi169.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-16T08:38:16.199 INFO:teuthology.orchestra.run.smithi169.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 35 MB/s | 2.0 MB 00:00 2024-09-16T08:38:16.225 INFO:teuthology.orchestra.run.smithi169.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-16T08:38:16.250 INFO:teuthology.orchestra.run.smithi169.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-16T08:38:16.284 INFO:teuthology.orchestra.run.smithi169.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.5 MB/s | 188 kB 00:00 2024-09-16T08:38:16.467 INFO:teuthology.orchestra.run.smithi169.stdout:(11/12): ceph-mgr-dashboard-19.3.0-4917.g8293d7 19 MB/s | 14 MB 00:00 2024-09-16T08:38:17.067 INFO:teuthology.orchestra.run.smithi169.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 827 kB/s | 863 kB 00:01 2024-09-16T08:38:17.072 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:17.072 INFO:teuthology.orchestra.run.smithi169.stdout:Total 10 MB/s | 19 MB 00:01 2024-09-16T08:38:17.125 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:38:17.146 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:38:17.146 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:38:17.408 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:38:17.409 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:38:17.616 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4917.g8293d73f.el 11/12 2024-09-16T08:38:17.683 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4917.g8293d73f.el9.noa 12/12 2024-09-16T08:38:18.015 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:38:18.093 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-16T08:38:18.128 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-16T08:38:18.182 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-16T08:38:18.213 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-16T08:38:18.314 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-16T08:38:18.440 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-16T08:38:18.524 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-16T08:38:18.576 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-16T08:38:18.605 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-16T08:38:18.637 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4917.g8293d73f.el9 10/12 2024-09-16T08:38:19.560 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4917.g8293d73f.el9.noa 12/12 2024-09-16T08:38:19.560 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4917.g8293d73f.el 1/12 2024-09-16T08:38:19.560 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4917.g8293d73f.el9.noa 2/12 2024-09-16T08:38:19.560 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4917.g8293d73f.el9 3/12 2024-09-16T08:38:19.560 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-16T08:38:19.560 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-16T08:38:19.560 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-16T08:38:19.560 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-16T08:38:19.561 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-16T08:38:19.561 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-16T08:38:19.561 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-16T08:38:19.561 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-16T08:38:19.881 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-16T08:38:19.881 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:19.881 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:38:19.881 INFO:teuthology.orchestra.run.smithi160.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T08:38:19.881 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:19.881 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:19.881 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:19.881 INFO:teuthology.orchestra.run.smithi160.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T08:38:19.881 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T08:38:19.882 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T08:38:19.882 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T08:38:19.882 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T08:38:19.882 INFO:teuthology.orchestra.run.smithi160.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T08:38:19.882 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T08:38:19.882 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T08:38:19.882 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:19.882 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:20.127 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T08:38:20.691 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:07 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:20.824 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:20.825 INFO:teuthology.orchestra.run.smithi160.stdout:======================================================================================= 2024-09-16T08:38:20.825 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-09-16T08:38:20.825 INFO:teuthology.orchestra.run.smithi160.stdout:======================================================================================= 2024-09-16T08:38:20.825 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:38:20.825 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 7.4 M 2024-09-16T08:38:20.825 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-09-16T08:38:20.825 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T08:38:20.826 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T08:38:20.826 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T08:38:20.826 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T08:38:20.826 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T08:38:20.826 INFO:teuthology.orchestra.run.smithi160.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T08:38:20.826 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T08:38:20.826 INFO:teuthology.orchestra.run.smithi160.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T08:38:20.826 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T08:38:20.826 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T08:38:20.826 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T08:38:20.826 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:20.826 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:38:20.826 INFO:teuthology.orchestra.run.smithi160.stdout:======================================================================================= 2024-09-16T08:38:20.827 INFO:teuthology.orchestra.run.smithi160.stdout:Install 12 Packages 2024-09-16T08:38:20.827 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:20.827 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 38 M 2024-09-16T08:38:20.827 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 212 M 2024-09-16T08:38:20.827 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:38:21.156 INFO:teuthology.orchestra.run.smithi160.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-16T08:38:21.215 INFO:teuthology.orchestra.run.smithi160.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 801 kB 00:00 2024-09-16T08:38:21.507 INFO:teuthology.orchestra.run.smithi160.stdout:(3/12): ceph-mgr-diskprediction-local-19.3.0-49 16 MB/s | 7.4 MB 00:00 2024-09-16T08:38:21.524 INFO:teuthology.orchestra.run.smithi160.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 89 kB/s | 33 kB 00:00 2024-09-16T08:38:21.716 INFO:teuthology.orchestra.run.smithi160.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 201 kB/s | 38 kB 00:00 2024-09-16T08:38:21.783 INFO:teuthology.orchestra.run.smithi160.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 66 kB/s | 18 kB 00:00 2024-09-16T08:38:21.837 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4917.g8293d73f.el 11/12 2024-09-16T08:38:21.891 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4917.g8293d73f.el9.noa 12/12 2024-09-16T08:38:22.141 INFO:teuthology.orchestra.run.smithi160.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.2 MB/s | 3.0 MB 00:00 2024-09-16T08:38:22.167 INFO:teuthology.orchestra.run.smithi160.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 638 kB/s | 245 kB 00:00 2024-09-16T08:38:22.308 INFO:teuthology.orchestra.run.smithi160.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.7 MB/s | 250 kB 00:00 2024-09-16T08:38:22.409 INFO:teuthology.orchestra.run.smithi160.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-09-16T08:38:22.551 INFO:teuthology.orchestra.run.smithi160.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 5.9 MB/s | 4.9 MB 00:00 2024-09-16T08:38:23.863 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4917.g8293d73f.el9.noa 12/12 2024-09-16T08:38:23.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4917.g8293d73f.el 1/12 2024-09-16T08:38:23.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4917.g8293d73f.el9.noa 2/12 2024-09-16T08:38:23.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4917.g8293d73f.el9 3/12 2024-09-16T08:38:23.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-16T08:38:23.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-16T08:38:23.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-16T08:38:23.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-16T08:38:23.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-16T08:38:23.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-16T08:38:23.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-16T08:38:23.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-16T08:38:23.876 INFO:teuthology.orchestra.run.smithi160.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-16T08:38:23.877 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:23.877 INFO:teuthology.orchestra.run.smithi160.stdout:Total 13 MB/s | 38 MB 00:03 2024-09-16T08:38:24.198 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-16T08:38:24.198 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:24.198 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:38:24.198 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T08:38:24.199 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:24.199 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:24.199 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:24.199 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T08:38:24.199 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T08:38:24.199 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T08:38:24.199 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T08:38:24.199 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T08:38:24.199 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T08:38:24.199 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T08:38:24.199 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T08:38:24.199 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:24.200 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:38:24.236 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:38:24.262 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:38:24.262 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:38:24.451 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T08:38:24.717 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:38:24.717 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:38:25.021 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:12 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:25.158 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:38:25.159 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-09-16T08:38:25.159 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T08:38:25.159 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-09-16T08:38:25.159 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:38:25.159 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 7.4 M 2024-09-16T08:38:25.159 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T08:38:25.159 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T08:38:25.159 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T08:38:25.160 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T08:38:25.160 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T08:38:25.160 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T08:38:25.160 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T08:38:25.160 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T08:38:25.160 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T08:38:25.160 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T08:38:25.160 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T08:38:25.160 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T08:38:25.160 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:25.160 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:38:25.160 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-09-16T08:38:25.160 INFO:teuthology.orchestra.run.smithi169.stdout:Install 12 Packages 2024-09-16T08:38:25.161 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:25.161 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 38 M 2024-09-16T08:38:25.161 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 212 M 2024-09-16T08:38:25.161 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:38:25.478 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:38:25.552 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T08:38:25.599 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T08:38:25.632 INFO:teuthology.orchestra.run.smithi169.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-16T08:38:25.682 INFO:teuthology.orchestra.run.smithi169.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.0 MB/s | 801 kB 00:00 2024-09-16T08:38:25.699 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T08:38:25.741 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T08:38:25.927 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T08:38:25.949 INFO:teuthology.orchestra.run.smithi169.stdout:(3/12): ceph-mgr-diskprediction-local-19.3.0-49 16 MB/s | 7.4 MB 00:00 2024-09-16T08:38:25.974 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T08:38:25.974 INFO:teuthology.orchestra.run.smithi169.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 96 kB/s | 33 kB 00:00 2024-09-16T08:38:26.051 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T08:38:26.175 INFO:teuthology.orchestra.run.smithi169.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 192 kB/s | 38 kB 00:00 2024-09-16T08:38:26.233 INFO:teuthology.orchestra.run.smithi169.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 64 kB/s | 18 kB 00:00 2024-09-16T08:38:26.700 INFO:teuthology.orchestra.run.smithi169.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 524 kB/s | 245 kB 00:00 2024-09-16T08:38:26.708 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T08:38:26.817 INFO:teuthology.orchestra.run.smithi169.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.6 MB/s | 3.0 MB 00:01 2024-09-16T08:38:26.837 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T08:38:26.934 INFO:teuthology.orchestra.run.smithi169.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-09-16T08:38:27.209 INFO:teuthology.orchestra.run.smithi169.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.8 MB/s | 4.9 MB 00:01 2024-09-16T08:38:27.585 INFO:teuthology.orchestra.run.smithi169.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.8 MB/s | 5.1 MB 00:00 2024-09-16T08:38:28.095 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T08:38:29.131 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T08:38:29.156 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4917.g8293d 12/12 2024-09-16T08:38:31.630 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4917.g8293d 12/12 2024-09-16T08:38:31.630 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4917.g8293d 1/12 2024-09-16T08:38:31.630 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T08:38:31.631 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T08:38:31.631 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T08:38:31.631 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T08:38:31.631 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T08:38:31.631 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T08:38:31.631 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T08:38:31.631 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T08:38:31.631 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T08:38:31.631 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T08:38:32.042 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T08:38:32.042 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:32.043 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:38:32.043 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:32.043 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T08:38:32.043 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T08:38:32.043 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T08:38:32.043 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T08:38:32.043 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T08:38:32.043 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T08:38:32.043 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T08:38:32.043 INFO:teuthology.orchestra.run.smithi160.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T08:38:32.043 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T08:38:32.043 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T08:38:32.043 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T08:38:32.044 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:32.044 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:32.473 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-mgr-rook 2024-09-16T08:38:32.801 INFO:teuthology.orchestra.run.smithi169.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.8 MB/s | 16 MB 00:05 2024-09-16T08:38:32.802 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:32.802 INFO:teuthology.orchestra.run.smithi169.stdout:Total 5.0 MB/s | 38 MB 00:07 2024-09-16T08:38:33.066 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:20 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:33.161 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:38:33.188 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:38:33.189 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:38:33.201 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:33.202 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-09-16T08:38:33.202 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:33.202 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:38:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 49 k 2024-09-16T08:38:33.202 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-09-16T08:38:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T08:38:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T08:38:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T08:38:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T08:38:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T08:38:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T08:38:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T08:38:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T08:38:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T08:38:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T08:38:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T08:38:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T08:38:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:33.204 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:38:33.204 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:33.204 INFO:teuthology.orchestra.run.smithi160.stdout:Install 13 Packages 2024-09-16T08:38:33.204 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:33.204 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 2.2 M 2024-09-16T08:38:33.204 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 26 M 2024-09-16T08:38:33.204 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:38:33.399 INFO:teuthology.orchestra.run.smithi160.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 180 kB/s | 19 kB 00:00 2024-09-16T08:38:33.424 INFO:teuthology.orchestra.run.smithi160.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 194 kB/s | 26 kB 00:00 2024-09-16T08:38:33.449 INFO:teuthology.orchestra.run.smithi160.stdout:(3/13): ceph-mgr-rook-19.3.0-4917.g8293d73f.el9 313 kB/s | 49 kB 00:00 2024-09-16T08:38:33.533 INFO:teuthology.orchestra.run.smithi160.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.6 MB/s | 222 kB 00:00 2024-09-16T08:38:33.558 INFO:teuthology.orchestra.run.smithi160.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-16T08:38:33.583 INFO:teuthology.orchestra.run.smithi160.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-16T08:38:33.625 INFO:teuthology.orchestra.run.smithi160.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 279 kB 00:00 2024-09-16T08:38:33.626 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:38:33.626 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:38:33.650 INFO:teuthology.orchestra.run.smithi160.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 349 kB/s | 32 kB 00:00 2024-09-16T08:38:33.676 INFO:teuthology.orchestra.run.smithi160.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 153 kB/s | 14 kB 00:00 2024-09-16T08:38:33.701 INFO:teuthology.orchestra.run.smithi160.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.5 MB/s | 59 kB 00:00 2024-09-16T08:38:33.726 INFO:teuthology.orchestra.run.smithi160.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.5 MB/s | 90 kB 00:00 2024-09-16T08:38:33.752 INFO:teuthology.orchestra.run.smithi160.stdout:(12/13): python3-google-auth-2.34.0-1.el9.noarc 1.7 MB/s | 220 kB 00:00 2024-09-16T08:38:33.785 INFO:teuthology.orchestra.run.smithi160.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 8.0 MB/s | 1.0 MB 00:00 2024-09-16T08:38:33.788 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:33.788 INFO:teuthology.orchestra.run.smithi160.stdout:Total 3.8 MB/s | 2.2 MB 00:00 2024-09-16T08:38:33.852 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:38:33.867 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:38:33.868 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:38:33.956 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:38:33.957 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:38:34.402 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:38:34.468 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:38:34.480 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T08:38:34.526 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T08:38:34.624 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T08:38:34.639 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T08:38:34.675 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T08:38:34.694 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T08:38:34.749 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T08:38:34.787 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T08:38:34.829 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T08:38:34.861 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T08:38:34.891 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T08:38:34.929 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T08:38:34.977 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T08:38:35.039 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T08:38:35.089 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T08:38:35.630 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T08:38:35.762 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T08:38:35.807 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T08:38:35.852 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T08:38:35.886 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T08:38:35.935 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T08:38:35.973 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-rook-2:19.3.0-4917.g8293d73f.el9.noarch 13/13 2024-09-16T08:38:36.764 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4917.g8293d73f.el9.noarch 13/13 2024-09-16T08:38:36.765 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4917.g8293d73f.el9.noarch 1/13 2024-09-16T08:38:36.765 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T08:38:36.765 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T08:38:36.765 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T08:38:36.765 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T08:38:36.765 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T08:38:36.765 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T08:38:36.765 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T08:38:36.765 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T08:38:36.765 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T08:38:36.766 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T08:38:36.766 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T08:38:37.012 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T08:38:37.104 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T08:38:37.104 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:37.104 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:38:37.104 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:37.104 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T08:38:37.104 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T08:38:37.104 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T08:38:37.105 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T08:38:37.105 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T08:38:37.105 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T08:38:37.105 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T08:38:37.105 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T08:38:37.105 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T08:38:37.105 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T08:38:37.105 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T08:38:37.105 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T08:38:37.105 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:37.106 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:37.384 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T08:38:37.953 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:24 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:38.059 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T08:38:38.089 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:38.090 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:38.090 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-09-16T08:38:38.090 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:38.090 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:38:38.090 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 154 k 2024-09-16T08:38:38.090 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-09-16T08:38:38.090 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T08:38:38.090 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T08:38:38.090 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T08:38:38.090 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T08:38:38.091 INFO:teuthology.orchestra.run.smithi160.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T08:38:38.091 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:38.091 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:38:38.091 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:38.091 INFO:teuthology.orchestra.run.smithi160.stdout:Install 6 Packages 2024-09-16T08:38:38.091 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:38.091 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4917.g8293d 12/12 2024-09-16T08:38:38.092 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 7.0 M 2024-09-16T08:38:38.092 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 33 M 2024-09-16T08:38:38.092 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:38:38.462 INFO:teuthology.orchestra.run.smithi160.stdout:(1/6): ceph-mgr-cephadm-19.3.0-4917.g8293d73f.e 710 kB/s | 154 kB 00:00 2024-09-16T08:38:38.512 INFO:teuthology.orchestra.run.smithi160.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-16T08:38:38.537 INFO:teuthology.orchestra.run.smithi160.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-16T08:38:38.562 INFO:teuthology.orchestra.run.smithi160.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-16T08:38:38.604 INFO:teuthology.orchestra.run.smithi160.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 690 kB/s | 248 kB 00:00 2024-09-16T08:38:40.835 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4917.g8293d 12/12 2024-09-16T08:38:40.835 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4917.g8293d 1/12 2024-09-16T08:38:40.835 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T08:38:40.835 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T08:38:40.835 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T08:38:40.835 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T08:38:40.835 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T08:38:40.835 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T08:38:40.835 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T08:38:40.836 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T08:38:40.836 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T08:38:40.836 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T08:38:41.057 INFO:teuthology.orchestra.run.smithi160.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.1 MB/s | 6.0 MB 00:02 2024-09-16T08:38:41.057 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:41.057 INFO:teuthology.orchestra.run.smithi160.stdout:Total 2.4 MB/s | 7.0 MB 00:02 2024-09-16T08:38:41.127 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:38:41.139 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:38:41.139 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:38:41.238 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:38:41.239 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:38:41.256 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T08:38:41.256 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:41.256 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:38:41.256 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:41.256 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T08:38:41.256 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T08:38:41.257 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T08:38:41.257 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T08:38:41.257 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T08:38:41.257 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T08:38:41.257 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T08:38:41.257 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T08:38:41.257 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T08:38:41.257 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T08:38:41.257 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T08:38:41.257 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:41.257 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:38:41.546 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:38:41.645 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr-rook 2024-09-16T08:38:41.721 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T08:38:41.773 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T08:38:42.229 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:29 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:42.323 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T08:38:42.365 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:38:42.366 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:42.366 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T08:38:42.366 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:42.366 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:38:42.366 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 49 k 2024-09-16T08:38:42.366 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T08:38:42.366 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T08:38:42.366 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T08:38:42.367 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T08:38:42.367 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T08:38:42.367 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T08:38:42.367 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T08:38:42.367 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T08:38:42.367 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T08:38:42.367 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T08:38:42.367 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T08:38:42.367 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T08:38:42.367 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T08:38:42.367 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:42.367 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:38:42.367 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:42.367 INFO:teuthology.orchestra.run.smithi169.stdout:Install 13 Packages 2024-09-16T08:38:42.368 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:42.368 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 2.2 M 2024-09-16T08:38:42.368 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 26 M 2024-09-16T08:38:42.368 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:38:42.409 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T08:38:42.476 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T08:38:42.502 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 6/6 2024-09-16T08:38:42.834 INFO:teuthology.orchestra.run.smithi169.stdout:(1/13): ceph-mgr-rook-19.3.0-4917.g8293d73f.el9 315 kB/s | 49 kB 00:00 2024-09-16T08:38:42.950 INFO:teuthology.orchestra.run.smithi169.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 94 kB/s | 26 kB 00:00 2024-09-16T08:38:42.967 INFO:teuthology.orchestra.run.smithi169.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 66 kB/s | 19 kB 00:00 2024-09-16T08:38:43.298 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 6/6 2024-09-16T08:38:43.299 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/6 2024-09-16T08:38:43.299 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T08:38:43.299 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T08:38:43.299 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T08:38:43.299 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T08:38:43.318 INFO:teuthology.orchestra.run.smithi169.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 432 kB/s | 159 kB 00:00 2024-09-16T08:38:43.343 INFO:teuthology.orchestra.run.smithi169.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 742 kB/s | 279 kB 00:00 2024-09-16T08:38:43.385 INFO:teuthology.orchestra.run.smithi169.stdout:(6/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 403 kB/s | 222 kB 00:00 2024-09-16T08:38:43.419 INFO:teuthology.orchestra.run.smithi169.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 428 kB/s | 32 kB 00:00 2024-09-16T08:38:43.444 INFO:teuthology.orchestra.run.smithi169.stdout:(8/13): python3-requests-oauthlib-1.3.0-12.el9. 426 kB/s | 54 kB 00:00 2024-09-16T08:38:43.469 INFO:teuthology.orchestra.run.smithi169.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 172 kB/s | 14 kB 00:00 2024-09-16T08:38:43.503 INFO:teuthology.orchestra.run.smithi169.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-16T08:38:43.519 INFO:teuthology.orchestra.run.smithi169.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 2.2 MB/s | 220 kB 00:00 2024-09-16T08:38:43.536 INFO:teuthology.orchestra.run.smithi169.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 2.8 MB/s | 90 kB 00:00 2024-09-16T08:38:43.578 INFO:teuthology.orchestra.run.smithi169.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.6 MB/s | 1.0 MB 00:00 2024-09-16T08:38:43.582 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:43.582 INFO:teuthology.orchestra.run.smithi169.stdout:Total 1.8 MB/s | 2.2 MB 00:01 2024-09-16T08:38:43.601 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T08:38:43.602 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:43.602 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:38:43.602 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:43.602 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T08:38:43.602 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T08:38:43.602 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T08:38:43.602 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T08:38:43.602 INFO:teuthology.orchestra.run.smithi160.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T08:38:43.602 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:43.602 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:43.652 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:38:43.668 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:38:43.668 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:38:43.757 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:38:43.759 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:38:43.853 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-fuse 2024-09-16T08:38:44.262 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:38:44.422 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:31 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:44.433 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T08:38:44.488 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T08:38:44.551 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T08:38:44.556 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:44.556 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:44.556 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-09-16T08:38:44.557 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:44.557 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:38:44.557 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 900 k 2024-09-16T08:38:44.557 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-09-16T08:38:44.557 INFO:teuthology.orchestra.run.smithi160.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T08:38:44.557 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:44.557 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:38:44.558 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:44.558 INFO:teuthology.orchestra.run.smithi160.stdout:Install 2 Packages 2024-09-16T08:38:44.558 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:44.558 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 979 k 2024-09-16T08:38:44.559 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 2.9 M 2024-09-16T08:38:44.559 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:38:44.589 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T08:38:44.630 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T08:38:44.731 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T08:38:44.790 INFO:teuthology.orchestra.run.smithi160.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 590 kB/s | 80 kB 00:00 2024-09-16T08:38:44.832 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T08:38:44.881 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T08:38:44.932 INFO:teuthology.orchestra.run.smithi160.stdout:(2/2): ceph-fuse-19.3.0-4917.g8293d73f.el9.x86_ 3.2 MB/s | 900 kB 00:00 2024-09-16T08:38:44.933 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:44.933 INFO:teuthology.orchestra.run.smithi160.stdout:Total 2.5 MB/s | 979 kB 00:00 2024-09-16T08:38:44.945 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:38:44.966 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:38:44.966 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:38:45.048 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:38:45.048 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:38:45.234 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:38:45.376 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T08:38:45.439 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 2/2 2024-09-16T08:38:45.590 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T08:38:45.637 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T08:38:45.671 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T08:38:45.719 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T08:38:45.741 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:19.3.0-4917.g8293d73f.el9.noarch 13/13 2024-09-16T08:38:46.068 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 2/2 2024-09-16T08:38:46.068 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 1/2 2024-09-16T08:38:46.362 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T08:38:46.362 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:46.362 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:38:46.362 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T08:38:46.362 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:46.363 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:46.493 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4917.g8293d73f.el9.noarch 13/13 2024-09-16T08:38:46.493 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4917.g8293d73f.el9.noarch 1/13 2024-09-16T08:38:46.493 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T08:38:46.494 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T08:38:46.494 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T08:38:46.494 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T08:38:46.494 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T08:38:46.494 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T08:38:46.494 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T08:38:46.494 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T08:38:46.494 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T08:38:46.494 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T08:38:46.494 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T08:38:46.655 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-volume 2024-09-16T08:38:46.832 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T08:38:46.832 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:46.833 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:38:46.833 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:46.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T08:38:46.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T08:38:46.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T08:38:46.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T08:38:46.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T08:38:46.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T08:38:46.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T08:38:46.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T08:38:46.833 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T08:38:46.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T08:38:46.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T08:38:46.834 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T08:38:46.834 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:46.834 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:38:47.120 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T08:38:47.240 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:34 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:47.382 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:47.383 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:47.383 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-09-16T08:38:47.383 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:47.383 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:38:47.383 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 285 k 2024-09-16T08:38:47.383 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:47.383 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:38:47.383 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:47.383 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2024-09-16T08:38:47.383 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:47.383 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 285 k 2024-09-16T08:38:47.384 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 1.3 M 2024-09-16T08:38:47.384 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:38:47.595 INFO:teuthology.orchestra.run.smithi160.stdout:ceph-volume-19.3.0-4917.g8293d73f.el9.noarch.rp 1.3 MB/s | 285 kB 00:00 2024-09-16T08:38:47.595 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:47.595 INFO:teuthology.orchestra.run.smithi160.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-16T08:38:47.596 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:38:47.604 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:38:47.604 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:38:47.632 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:38:47.632 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:38:47.699 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:34 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:47.837 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:38:47.838 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:47.838 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T08:38:47.838 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:47.838 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:38:47.838 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 154 k 2024-09-16T08:38:47.838 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T08:38:47.838 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T08:38:47.838 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T08:38:47.839 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T08:38:47.839 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T08:38:47.839 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T08:38:47.839 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:47.839 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:38:47.839 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:47.839 INFO:teuthology.orchestra.run.smithi169.stdout:Install 6 Packages 2024-09-16T08:38:47.839 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:47.840 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 7.0 M 2024-09-16T08:38:47.840 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 33 M 2024-09-16T08:38:47.840 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:38:47.848 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:38:47.922 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-volume-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T08:38:47.938 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-volume-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T08:38:47.938 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:38:47.938 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T08:38:47.939 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:48.172 INFO:teuthology.orchestra.run.smithi169.stdout:(1/6): ceph-mgr-cephadm-19.3.0-4917.g8293d73f.e 681 kB/s | 154 kB 00:00 2024-09-16T08:38:48.205 INFO:teuthology.orchestra.run.smithi169.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 954 kB/s | 248 kB 00:00 2024-09-16T08:38:48.256 INFO:teuthology.orchestra.run.smithi169.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-16T08:38:48.272 INFO:teuthology.orchestra.run.smithi169.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 960 kB/s | 58 kB 00:00 2024-09-16T08:38:48.298 INFO:teuthology.orchestra.run.smithi169.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.7 MB/s | 74 kB 00:00 2024-09-16T08:38:48.456 INFO:teuthology.orchestra.run.smithi169.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-09-16T08:38:48.458 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:48.458 INFO:teuthology.orchestra.run.smithi169.stdout:Total 11 MB/s | 7.0 MB 00:00 2024-09-16T08:38:48.529 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:38:48.540 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:38:48.540 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:38:48.641 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:38:48.641 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:38:48.949 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:38:48.977 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-volume-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T08:38:48.977 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:48.977 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:38:48.977 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:48.977 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:48.978 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:49.133 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T08:38:49.183 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T08:38:49.296 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install librados-devel 2024-09-16T08:38:49.719 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T08:38:49.820 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T08:38:49.884 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:36 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:49.888 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T08:38:49.912 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 6/6 2024-09-16T08:38:50.020 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:50.021 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:50.021 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repo Size 2024-09-16T08:38:50.021 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:50.022 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:38:50.022 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 126 k 2024-09-16T08:38:50.022 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:50.022 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:38:50.022 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:50.022 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2024-09-16T08:38:50.022 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:50.022 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 126 k 2024-09-16T08:38:50.022 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 456 k 2024-09-16T08:38:50.022 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:38:50.242 INFO:teuthology.orchestra.run.smithi160.stdout:librados-devel-19.3.0-4917.g8293d73f.el9.x86_64 577 kB/s | 126 kB 00:00 2024-09-16T08:38:50.243 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:50.243 INFO:teuthology.orchestra.run.smithi160.stdout:Total 573 kB/s | 126 kB 00:00 2024-09-16T08:38:50.243 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:38:50.248 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:38:50.249 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:38:50.314 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:38:50.314 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:38:50.464 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:38:50.565 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librados-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:38:50.703 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 6/6 2024-09-16T08:38:50.703 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/6 2024-09-16T08:38:50.703 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T08:38:50.704 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T08:38:50.704 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T08:38:50.704 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T08:38:50.981 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T08:38:50.981 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:50.982 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:38:50.982 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:50.982 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T08:38:50.982 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T08:38:50.982 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T08:38:50.982 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T08:38:50.982 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T08:38:50.982 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:50.982 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:38:51.042 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:38:51.272 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-fuse 2024-09-16T08:38:51.303 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:38:51.303 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:51.303 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:38:51.303 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:38:51.303 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:51.303 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:51.593 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install libcephfs2 2024-09-16T08:38:51.848 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:38 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:51.984 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:38:51.984 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:51.984 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T08:38:51.984 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:51.984 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:38:51.984 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 900 k 2024-09-16T08:38:51.985 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T08:38:51.985 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T08:38:51.985 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:51.985 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:38:51.985 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:51.985 INFO:teuthology.orchestra.run.smithi169.stdout:Install 2 Packages 2024-09-16T08:38:51.985 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:51.986 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 979 k 2024-09-16T08:38:51.986 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 2.9 M 2024-09-16T08:38:51.986 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:38:52.117 INFO:teuthology.orchestra.run.smithi169.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 800 kB/s | 80 kB 00:00 2024-09-16T08:38:52.177 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:39 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:52.270 INFO:teuthology.orchestra.run.smithi160.stdout:Package libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:38:52.316 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:52.317 INFO:teuthology.orchestra.run.smithi169.stdout:(2/2): ceph-fuse-19.3.0-4917.g8293d73f.el9.x86_ 2.9 MB/s | 900 kB 00:00 2024-09-16T08:38:52.317 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-09-16T08:38:52.317 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:52.318 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:52.318 INFO:teuthology.orchestra.run.smithi169.stdout:Total 2.9 MB/s | 979 kB 00:00 2024-09-16T08:38:52.329 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:38:52.351 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:38:52.351 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:38:52.397 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install libcephfs-devel 2024-09-16T08:38:52.434 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:38:52.434 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:38:52.632 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:38:52.779 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T08:38:52.849 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 2/2 2024-09-16T08:38:52.964 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:39 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:53.097 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:53.098 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:53.098 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repo Size 2024-09-16T08:38:53.098 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:53.098 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:38:53.098 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 32 k 2024-09-16T08:38:53.099 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:53.099 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:38:53.099 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:53.099 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2024-09-16T08:38:53.099 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:53.099 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 32 k 2024-09-16T08:38:53.099 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 142 k 2024-09-16T08:38:53.099 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:38:53.246 INFO:teuthology.orchestra.run.smithi160.stdout:libcephfs-devel-19.3.0-4917.g8293d73f.el9.x86_6 216 kB/s | 32 kB 00:00 2024-09-16T08:38:53.247 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:53.247 INFO:teuthology.orchestra.run.smithi160.stdout:Total 214 kB/s | 32 kB 00:00 2024-09-16T08:38:53.247 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:38:53.250 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:38:53.250 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:38:53.269 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:38:53.270 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:38:53.360 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:38:53.477 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:38:53.543 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 2/2 2024-09-16T08:38:53.543 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 1/2 2024-09-16T08:38:53.849 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T08:38:53.849 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:53.849 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:38:53.849 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T08:38:53.850 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:53.850 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:38:53.861 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:38:54.108 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:38:54.109 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:54.109 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:38:54.109 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:38:54.109 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:54.109 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:54.165 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-volume 2024-09-16T08:38:54.329 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install librados2 2024-09-16T08:38:54.752 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:41 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:54.892 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:38:54.892 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:54.892 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T08:38:54.893 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:54.893 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:38:54.893 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 285 k 2024-09-16T08:38:54.893 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:54.893 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:38:54.893 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:54.893 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-09-16T08:38:54.893 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:54.893 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 285 k 2024-09-16T08:38:54.893 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 1.3 M 2024-09-16T08:38:54.893 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:38:54.900 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:41 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:54.989 INFO:teuthology.orchestra.run.smithi160.stdout:Package librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:38:55.033 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:55.033 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-09-16T08:38:55.033 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:55.117 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install librbd1 2024-09-16T08:38:55.121 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-volume-19.3.0-4917.g8293d73f.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-16T08:38:55.122 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:55.122 INFO:teuthology.orchestra.run.smithi169.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-16T08:38:55.123 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:38:55.131 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:38:55.131 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:38:55.160 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:38:55.160 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:38:55.383 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:38:55.448 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-volume-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T08:38:55.465 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-volume-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T08:38:55.465 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:38:55.465 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T08:38:55.465 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:55.691 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:42 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:55.782 INFO:teuthology.orchestra.run.smithi160.stdout:Package librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:38:55.825 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:55.826 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-09-16T08:38:55.827 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:55.915 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install python3-rados 2024-09-16T08:38:56.484 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:43 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:56.492 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-volume-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T08:38:56.492 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:56.492 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:38:56.492 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T08:38:56.492 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:56.492 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:38:56.576 INFO:teuthology.orchestra.run.smithi160.stdout:Package python3-rados-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:38:56.619 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:56.619 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-09-16T08:38:56.620 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:56.700 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install python3-rgw 2024-09-16T08:38:56.830 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install librados-devel 2024-09-16T08:38:57.272 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:44 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:57.364 INFO:teuthology.orchestra.run.smithi160.stdout:Package python3-rgw-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:38:57.407 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:57.408 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-09-16T08:38:57.408 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:57.489 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:44 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:57.499 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install python3-cephfs 2024-09-16T08:38:57.622 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:38:57.623 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:57.623 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2024-09-16T08:38:57.623 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:57.623 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:38:57.623 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 126 k 2024-09-16T08:38:57.623 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:57.623 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:38:57.623 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:38:57.623 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-09-16T08:38:57.623 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:57.624 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 126 k 2024-09-16T08:38:57.624 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 456 k 2024-09-16T08:38:57.624 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:38:57.818 INFO:teuthology.orchestra.run.smithi169.stdout:librados-devel-19.3.0-4917.g8293d73f.el9.x86_64 653 kB/s | 126 kB 00:00 2024-09-16T08:38:57.818 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:57.819 INFO:teuthology.orchestra.run.smithi169.stdout:Total 648 kB/s | 126 kB 00:00 2024-09-16T08:38:57.819 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:38:57.824 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:38:57.824 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:38:57.889 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:38:57.889 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:38:58.039 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:38:58.069 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:45 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:58.148 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:38:58.156 INFO:teuthology.orchestra.run.smithi160.stdout:Package python3-cephfs-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:38:58.199 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:58.200 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-09-16T08:38:58.200 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:58.279 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install python3-rbd 2024-09-16T08:38:58.569 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:38:58.814 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:38:58.814 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:58.814 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:38:58.814 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:38:58.814 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:38:58.814 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:38:58.845 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:45 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:58.937 INFO:teuthology.orchestra.run.smithi160.stdout:Package python3-rbd-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:38:58.980 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:58.981 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2024-09-16T08:38:58.981 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:38:59.067 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install rbd-fuse 2024-09-16T08:38:59.149 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install libcephfs2 2024-09-16T08:38:59.646 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:46 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:59.737 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:46 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:38:59.789 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:38:59.790 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:59.790 INFO:teuthology.orchestra.run.smithi160.stdout: Package Architecture Version Repository Size 2024-09-16T08:38:59.790 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:59.791 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:38:59.791 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 85 k 2024-09-16T08:38:59.791 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:59.791 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:38:59.791 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:38:59.791 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2024-09-16T08:38:59.791 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:38:59.791 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 85 k 2024-09-16T08:38:59.791 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 231 k 2024-09-16T08:38:59.791 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:38:59.834 INFO:teuthology.orchestra.run.smithi169.stdout:Package libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:38:59.880 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:38:59.881 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T08:38:59.881 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:38:59.959 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install libcephfs-devel 2024-09-16T08:38:59.974 INFO:teuthology.orchestra.run.smithi160.stdout:rbd-fuse-19.3.0-4917.g8293d73f.el9.x86_64.rpm 465 kB/s | 85 kB 00:00 2024-09-16T08:38:59.976 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:38:59.976 INFO:teuthology.orchestra.run.smithi160.stdout:Total 459 kB/s | 85 kB 00:00 2024-09-16T08:38:59.976 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:38:59.988 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:38:59.988 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:39:00.065 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:39:00.066 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:39:00.262 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:39:00.373 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:00.537 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:47 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:39:00.671 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:39:00.671 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:39:00.671 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2024-09-16T08:39:00.671 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:39:00.671 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:39:00.671 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 32 k 2024-09-16T08:39:00.671 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:00.672 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:39:00.672 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:39:00.672 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-09-16T08:39:00.672 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:00.672 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 32 k 2024-09-16T08:39:00.672 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 142 k 2024-09-16T08:39:00.672 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:39:00.777 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:00.822 INFO:teuthology.orchestra.run.smithi169.stdout:libcephfs-devel-19.3.0-4917.g8293d73f.el9.x86_6 213 kB/s | 32 kB 00:00 2024-09-16T08:39:00.822 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:39:00.822 INFO:teuthology.orchestra.run.smithi169.stdout:Total 211 kB/s | 32 kB 00:00 2024-09-16T08:39:00.822 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:39:00.826 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:39:00.826 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:39:00.850 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:39:00.850 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:39:00.929 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:39:01.042 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:01.043 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:39:01.043 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:39:01.043 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:39:01.043 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:39:01.043 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:39:01.053 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:01.269 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install rbd-mirror 2024-09-16T08:39:01.454 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:01.695 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:01.695 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:01.696 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:39:01.696 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:39:01.696 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:01.696 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:39:01.852 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:48 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:39:01.908 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install librados2 2024-09-16T08:39:01.987 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:39:01.988 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:39:01.988 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repo Size 2024-09-16T08:39:01.988 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:39:01.989 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:39:01.989 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 3.2 M 2024-09-16T08:39:01.989 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:39:01.989 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:39:01.989 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:39:01.989 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2024-09-16T08:39:01.989 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:39:01.989 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 3.2 M 2024-09-16T08:39:01.989 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 14 M 2024-09-16T08:39:01.989 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:39:02.328 INFO:teuthology.orchestra.run.smithi160.stdout:rbd-mirror-19.3.0-4917.g8293d73f.el9.x86_64.rpm 9.4 MB/s | 3.2 MB 00:00 2024-09-16T08:39:02.329 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:39:02.329 INFO:teuthology.orchestra.run.smithi160.stdout:Total 9.4 MB/s | 3.2 MB 00:00 2024-09-16T08:39:02.329 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:39:02.338 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:39:02.339 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:39:02.420 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:39:02.421 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:39:02.481 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:49 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:39:02.571 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:39:02.615 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:39:02.615 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T08:39:02.616 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:39:02.700 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install librbd1 2024-09-16T08:39:02.876 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:39:02.946 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-mirror-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:02.971 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:02.971 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:39:02.972 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T08:39:02.972 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T08:39:02.972 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T08:39:02.972 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:39:03.275 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:50 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:39:03.366 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:39:03.410 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:39:03.411 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T08:39:03.411 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:39:03.494 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install python3-rados 2024-09-16T08:39:04.060 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:51 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:39:04.101 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-mirror-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:04.102 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:39:04.102 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:39:04.102 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:39:04.102 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:39:04.102 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:39:04.150 INFO:teuthology.orchestra.run.smithi169.stdout:Package python3-rados-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:39:04.194 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:39:04.195 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T08:39:04.195 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:39:04.276 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install python3-rgw 2024-09-16T08:39:04.364 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install rbd-nbd 2024-09-16T08:39:04.842 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:51 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:39:04.934 INFO:teuthology.orchestra.run.smithi169.stdout:Package python3-rgw-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:39:04.958 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:51 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:39:04.977 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:39:04.978 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T08:39:04.978 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:39:05.069 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install python3-cephfs 2024-09-16T08:39:05.093 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-09-16T08:39:05.094 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:39:05.094 INFO:teuthology.orchestra.run.smithi160.stdout: Package Architecture Version Repository Size 2024-09-16T08:39:05.094 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:39:05.094 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-09-16T08:39:05.095 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 171 k 2024-09-16T08:39:05.095 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:39:05.095 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-09-16T08:39:05.095 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-09-16T08:39:05.095 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2024-09-16T08:39:05.095 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:39:05.095 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 171 k 2024-09-16T08:39:05.095 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 490 k 2024-09-16T08:39:05.095 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-09-16T08:39:05.306 INFO:teuthology.orchestra.run.smithi160.stdout:rbd-nbd-19.3.0-4917.g8293d73f.el9.x86_64.rpm 808 kB/s | 171 kB 00:00 2024-09-16T08:39:05.307 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:39:05.307 INFO:teuthology.orchestra.run.smithi160.stdout:Total 803 kB/s | 171 kB 00:00 2024-09-16T08:39:05.307 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-09-16T08:39:05.315 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-09-16T08:39:05.315 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-09-16T08:39:05.380 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-09-16T08:39:05.380 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-09-16T08:39:05.535 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-09-16T08:39:05.636 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:52 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:39:05.654 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-nbd-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:05.726 INFO:teuthology.orchestra.run.smithi169.stdout:Package python3-cephfs-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:39:05.770 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:39:05.772 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T08:39:05.772 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:39:05.856 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install python3-rbd 2024-09-16T08:39:06.089 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:06.381 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-nbd-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:06.382 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:39:06.382 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-09-16T08:39:06.382 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:39:06.382 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:39:06.382 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-09-16T08:39:06.430 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:53 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:39:06.521 INFO:teuthology.orchestra.run.smithi169.stdout:Package python3-rbd-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T08:39:06.565 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:39:06.565 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T08:39:06.565 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:39:06.649 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install rbd-fuse 2024-09-16T08:39:06.654 DEBUG:teuthology.parallel:result is None 2024-09-16T08:39:07.223 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:54 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:39:07.361 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:39:07.362 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:39:07.362 INFO:teuthology.orchestra.run.smithi169.stdout: Package Architecture Version Repository Size 2024-09-16T08:39:07.362 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:39:07.362 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:39:07.362 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 85 k 2024-09-16T08:39:07.362 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:07.362 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:39:07.362 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:39:07.363 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-09-16T08:39:07.363 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:07.363 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 85 k 2024-09-16T08:39:07.363 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 231 k 2024-09-16T08:39:07.363 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:39:07.562 INFO:teuthology.orchestra.run.smithi169.stdout:rbd-fuse-19.3.0-4917.g8293d73f.el9.x86_64.rpm 430 kB/s | 85 kB 00:00 2024-09-16T08:39:07.562 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:39:07.563 INFO:teuthology.orchestra.run.smithi169.stdout:Total 426 kB/s | 85 kB 00:00 2024-09-16T08:39:07.563 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:39:07.570 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:39:07.570 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:39:07.635 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:39:07.635 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:39:07.836 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:39:07.956 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:08.346 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:08.617 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:08.617 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:08.617 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:39:08.617 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:39:08.617 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:08.618 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:39:08.930 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install rbd-mirror 2024-09-16T08:39:09.517 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:56 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:39:09.658 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:39:09.659 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:39:09.659 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2024-09-16T08:39:09.659 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:39:09.659 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:39:09.660 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 3.2 M 2024-09-16T08:39:09.660 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:09.660 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:39:09.660 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:39:09.660 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-09-16T08:39:09.660 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:09.660 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 3.2 M 2024-09-16T08:39:09.660 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 14 M 2024-09-16T08:39:09.660 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:39:10.004 INFO:teuthology.orchestra.run.smithi169.stdout:rbd-mirror-19.3.0-4917.g8293d73f.el9.x86_64.rpm 9.3 MB/s | 3.2 MB 00:00 2024-09-16T08:39:10.004 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:39:10.005 INFO:teuthology.orchestra.run.smithi169.stdout:Total 9.2 MB/s | 3.2 MB 00:00 2024-09-16T08:39:10.005 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:39:10.014 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:39:10.014 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:39:10.098 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:39:10.098 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:39:10.547 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:39:10.613 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:10.639 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:10.639 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:39:10.639 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T08:39:10.639 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T08:39:10.640 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T08:39:10.640 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:11.764 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:39:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:39:11.764 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:11.764 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:39:12.018 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install rbd-nbd 2024-09-16T08:39:12.602 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:59 ago on Mon 16 Sep 2024 08:37:13 AM UTC. 2024-09-16T08:39:12.737 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T08:39:12.738 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:39:12.738 INFO:teuthology.orchestra.run.smithi169.stdout: Package Architecture Version Repository Size 2024-09-16T08:39:12.738 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:39:12.738 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T08:39:12.738 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 171 k 2024-09-16T08:39:12.738 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:12.738 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T08:39:12.738 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T08:39:12.738 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-09-16T08:39:12.738 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:12.739 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 171 k 2024-09-16T08:39:12.739 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 490 k 2024-09-16T08:39:12.739 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T08:39:12.949 INFO:teuthology.orchestra.run.smithi169.stdout:rbd-nbd-19.3.0-4917.g8293d73f.el9.x86_64.rpm 813 kB/s | 171 kB 00:00 2024-09-16T08:39:12.950 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:39:12.950 INFO:teuthology.orchestra.run.smithi169.stdout:Total 806 kB/s | 171 kB 00:00 2024-09-16T08:39:12.950 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T08:39:12.958 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T08:39:12.958 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T08:39:13.022 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T08:39:13.023 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T08:39:13.176 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T08:39:13.271 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:13.691 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:13.960 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T08:39:13.960 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:13.960 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T08:39:13.960 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T08:39:13.960 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:39:13.960 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T08:39:14.239 DEBUG:teuthology.parallel:result is None 2024-09-16T08:39:14.240 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:39:14.388 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T08:39:14.414 INFO:teuthology.orchestra.run.smithi160.stdout:19.3.0-4917.g8293d73f.el9 2024-09-16T08:39:14.414 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4917.g8293d73f.el9 2024-09-16T08:39:14.415 INFO:teuthology.task.install:The correct ceph version 19.3.0-4917.g8293d73f is installed. 2024-09-16T08:39:14.417 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:39:14.556 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T08:39:14.577 INFO:teuthology.orchestra.run.smithi169.stdout:19.3.0-4917.g8293d73f.el9 2024-09-16T08:39:14.578 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4917.g8293d73f.el9 2024-09-16T08:39:14.578 INFO:teuthology.task.install:The correct ceph version 19.3.0-4917.g8293d73f is installed. 2024-09-16T08:39:14.579 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T08:39:14.580 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:39:14.580 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T08:39:14.611 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T08:39:14.611 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T08:39:14.645 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T08:39:14.645 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:39:14.645 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T08:39:14.681 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T08:39:14.749 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T08:39:14.749 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T08:39:14.773 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T08:39:14.839 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T08:39:14.839 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:39:14.839 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T08:39:14.867 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T08:39:14.933 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T08:39:14.934 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T08:39:14.959 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T08:39:15.024 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T08:39:15.024 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:39:15.024 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T08:39:15.051 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T08:39:15.118 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T08:39:15.118 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T08:39:15.142 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T08:39:15.207 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-16T08:39:15.308 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', 'CEPHADM_STRAY_DAEMON'], 'log-only-match': ['CEPHADM_'], 'mon_bind_msgr2': False, 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705', 'cephadm_mode': 'cephadm-package'} 2024-09-16T08:39:15.308 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:39:15.309 INFO:tasks.cephadm:Cluster fsid is 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:39:15.309 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-16T08:39:15.309 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.160:6789]', 'mon.c': '[v1:172.21.15.160:6790]', 'mon.b': '[v1:172.21.15.169:6789]'} 2024-09-16T08:39:15.309 INFO:tasks.cephadm:First mon is mon.a on smithi160 2024-09-16T08:39:15.309 INFO:tasks.cephadm:First mgr is y 2024-09-16T08:39:15.309 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-16T08:39:15.310 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname $(hostname -s) 2024-09-16T08:39:15.336 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname $(hostname -s) 2024-09-16T08:39:15.371 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 on all hosts... 2024-09-16T08:39:15.371 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 pull 2024-09-16T08:39:15.379 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 pull 2024-09-16T08:39:16.989 INFO:teuthology.orchestra.run.smithi160.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705... 2024-09-16T08:39:17.076 INFO:teuthology.orchestra.run.smithi169.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705... 2024-09-16T08:39:39.237 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-09-16T08:39:39.238 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_version": "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)", 2024-09-16T08:39:39.238 INFO:teuthology.orchestra.run.smithi160.stdout: "image_id": "425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c", 2024-09-16T08:39:39.238 INFO:teuthology.orchestra.run.smithi160.stdout: "repo_digests": [ 2024-09-16T08:39:39.238 INFO:teuthology.orchestra.run.smithi160.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce" 2024-09-16T08:39:39.238 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-09-16T08:39:39.238 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-09-16T08:39:44.295 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-09-16T08:39:44.295 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_version": "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)", 2024-09-16T08:39:44.295 INFO:teuthology.orchestra.run.smithi169.stdout: "image_id": "425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c", 2024-09-16T08:39:44.295 INFO:teuthology.orchestra.run.smithi169.stdout: "repo_digests": [ 2024-09-16T08:39:44.295 INFO:teuthology.orchestra.run.smithi169.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce" 2024-09-16T08:39:44.295 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2024-09-16T08:39:44.295 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-09-16T08:39:44.326 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /etc/ceph 2024-09-16T08:39:44.360 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph 2024-09-16T08:39:44.390 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 777 /etc/ceph 2024-09-16T08:39:44.429 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /etc/ceph 2024-09-16T08:39:44.457 INFO:tasks.cephadm:Writing seed config... 2024-09-16T08:39:44.459 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-16T08:39:44.459 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-16T08:39:44.459 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-16T08:39:44.459 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-09-16T08:39:44.459 INFO:tasks.cephadm: override: [global] ms bind msgr2 = False 2024-09-16T08:39:44.459 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-16T08:39:44.459 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-16T08:39:44.459 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-16T08:39:44.459 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-16T08:39:44.459 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-16T08:39:44.459 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-16T08:39:44.460 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-16T08:39:44.460 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-09-16T08:39:44.460 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-09-16T08:39:44.460 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-16T08:39:44.460 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:39:44.460 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-16T08:39:44.487 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 27cca2fc-7407-11ef-bceb-c7b262605968 mon election default strategy = 1 ms bind msgr2 = False ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-09-16T08:39:44.488 DEBUG:teuthology.orchestra.run.smithi160:mon.a> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mon.a.service 2024-09-16T08:39:44.530 DEBUG:teuthology.orchestra.run.smithi160:mgr.y> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mgr.y.service 2024-09-16T08:39:44.573 INFO:tasks.cephadm:Bootstrapping... 2024-09-16T08:39:44.573 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 -v bootstrap --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.160:6789]' --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-16T08:39:44.778 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:39:44.778 INFO:teuthology.orchestra.run.smithi160.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705', '-v', 'bootstrap', '--fsid', '27cca2fc-7407-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-addrv', '[v1:172.21.15.160:6789]', '--skip-admin-label'] 2024-09-16T08:39:44.778 INFO:teuthology.orchestra.run.smithi160.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-16T08:39:44.779 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying podman|docker is present... 2024-09-16T08:39:44.803 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stdout 5.2.2 2024-09-16T08:39:44.804 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying lvm2 is present... 2024-09-16T08:39:44.804 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying time synchronization is in place... 2024-09-16T08:39:44.811 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T08:39:44.811 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T08:39:44.817 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T08:39:44.818 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-09-16T08:39:44.824 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout enabled 2024-09-16T08:39:44.830 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout active 2024-09-16T08:39:44.830 INFO:teuthology.orchestra.run.smithi160.stdout:Unit chronyd.service is enabled and running 2024-09-16T08:39:44.830 INFO:teuthology.orchestra.run.smithi160.stdout:Repeating the final host check... 2024-09-16T08:39:44.853 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stdout 5.2.2 2024-09-16T08:39:44.853 INFO:teuthology.orchestra.run.smithi160.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-16T08:39:44.853 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl is present 2024-09-16T08:39:44.853 INFO:teuthology.orchestra.run.smithi160.stdout:lvcreate is present 2024-09-16T08:39:44.859 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T08:39:44.860 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T08:39:44.866 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T08:39:44.866 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-09-16T08:39:44.872 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout enabled 2024-09-16T08:39:44.879 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout active 2024-09-16T08:39:44.879 INFO:teuthology.orchestra.run.smithi160.stdout:Unit chronyd.service is enabled and running 2024-09-16T08:39:44.879 INFO:teuthology.orchestra.run.smithi160.stdout:Host looks OK 2024-09-16T08:39:44.879 INFO:teuthology.orchestra.run.smithi160.stdout:Cluster fsid: 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:39:44.879 INFO:teuthology.orchestra.run.smithi160.stdout:Acquiring lock 140242375674224 on /run/cephadm/27cca2fc-7407-11ef-bceb-c7b262605968.lock 2024-09-16T08:39:44.879 INFO:teuthology.orchestra.run.smithi160.stdout:Lock 140242375674224 acquired on /run/cephadm/27cca2fc-7407-11ef-bceb-c7b262605968.lock 2024-09-16T08:39:44.880 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying IP 172.21.15.160 port 6789 ... 2024-09-16T08:39:44.880 INFO:teuthology.orchestra.run.smithi160.stdout:Base mon IP(s) is [172.21.15.160:6789], mon addrv is [v1:172.21.15.160:6789] 2024-09-16T08:39:44.883 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.160 metric 100 2024-09-16T08:39:44.883 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.160 metric 100 2024-09-16T08:39:44.885 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-16T08:39:44.886 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-16T08:39:44.886 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-16T08:39:44.889 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-16T08:39:44.889 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-16T08:39:44.889 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T08:39:44.889 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-16T08:39:44.889 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7f5b/64 scope link noprefixroute 2024-09-16T08:39:44.890 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T08:39:44.890 INFO:teuthology.orchestra.run.smithi160.stdout:Mon IP `172.21.15.160` is in CIDR network `172.21.0.0/20` 2024-09-16T08:39:44.890 INFO:teuthology.orchestra.run.smithi160.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20'] 2024-09-16T08:39:44.891 INFO:teuthology.orchestra.run.smithi160.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-16T08:39:44.891 INFO:teuthology.orchestra.run.smithi160.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705... 2024-09-16T08:39:45.323 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stdout 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c 2024-09-16T08:39:45.323 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705... 2024-09-16T08:39:45.323 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Getting image source signatures 2024-09-16T08:39:45.323 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Copying blob sha256:7b1291ae133cf24c065f94db08f06624ab089b121831fa7f7f15d46063d39776 2024-09-16T08:39:45.323 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-16T08:39:45.323 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Copying config sha256:425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c 2024-09-16T08:39:45.323 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-16T08:39:46.276 INFO:teuthology.orchestra.run.smithi160.stdout:ceph: stdout ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev) 2024-09-16T08:39:46.276 INFO:teuthology.orchestra.run.smithi160.stdout:Ceph version: ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev) 2024-09-16T08:39:46.277 INFO:teuthology.orchestra.run.smithi160.stdout:Extracting ceph user uid/gid from container image... 2024-09-16T08:39:47.599 INFO:teuthology.orchestra.run.smithi160.stdout:stat: stdout 167 167 2024-09-16T08:39:47.599 INFO:teuthology.orchestra.run.smithi160.stdout:Creating initial keys... 2024-09-16T08:39:48.645 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph-authtool: stdout AQDT7udmAFCoORAADLUiso3eapDWx4BbYZT/fQ== 2024-09-16T08:39:49.559 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph-authtool: stdout AQDU7udmXYBjOxAAhWdJipR6TU9py1avkozXQA== 2024-09-16T08:39:50.514 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph-authtool: stdout AQDV7udml2ZLOBAAMHLz8B7xc57vPXvJ04zvhw== 2024-09-16T08:39:50.516 INFO:teuthology.orchestra.run.smithi160.stdout:Creating initial monmap... 2024-09-16T08:39:51.479 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T08:39:51.479 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-16T08:39:51.479 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:39:51.479 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T08:39:51.479 INFO:teuthology.orchestra.run.smithi160.stdout:monmaptool for a [v1:172.21.15.160:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T08:39:51.479 INFO:teuthology.orchestra.run.smithi160.stdout:setting min_mon_release = quincy 2024-09-16T08:39:51.479 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: set fsid to 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:39:51.479 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T08:39:51.479 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:39:51.480 INFO:teuthology.orchestra.run.smithi160.stdout:Creating mon... 2024-09-16T08:39:52.676 INFO:teuthology.orchestra.run.smithi160.stdout:create mon.a on 2024-09-16T08:39:52.879 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-16T08:39:53.022 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-16T08:39:53.181 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-27cca2fc-7407-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-27cca2fc-7407-11ef-bceb-c7b262605968.target. 2024-09-16T08:39:53.181 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-27cca2fc-7407-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-27cca2fc-7407-11ef-bceb-c7b262605968.target. 2024-09-16T08:39:53.426 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mon.a 2024-09-16T08:39:53.426 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Failed to reset failed state of unit ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-16T08:39:53.583 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-27cca2fc-7407-11ef-bceb-c7b262605968.target.wants/ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-27cca2fc-7407-11ef-bceb-c7b262605968@.service. 2024-09-16T08:39:54.102 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:54 smithi160 ceph-mon[29668]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T08:39:54.102 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:54 smithi160 bash[29631]: 511bef551290098faf2bbe56777833282856ee5d69cb03cb04be29645871ff5f 2024-09-16T08:39:54.116 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T08:39:54.116 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout disabled 2024-09-16T08:39:54.123 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T08:39:54.123 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-09-16T08:39:54.123 INFO:teuthology.orchestra.run.smithi160.stdout:firewalld.service is not enabled 2024-09-16T08:39:54.123 INFO:teuthology.orchestra.run.smithi160.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T08:39:54.123 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mon to start... 2024-09-16T08:39:54.123 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mon... 2024-09-16T08:39:54.356 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:54 smithi160 systemd[1]: Started Ceph mon.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:39:55.149 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout cluster: 2024-09-16T08:39:55.149 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout id: 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:39:55.149 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-16T08:39:55.149 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-09-16T08:39:55.149 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout services: 2024-09-16T08:39:55.150 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.534328s) 2024-09-16T08:39:55.150 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-16T08:39:55.150 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-16T08:39:55.150 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-09-16T08:39:55.150 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout data: 2024-09-16T08:39:55.150 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-16T08:39:55.150 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-16T08:39:55.150 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-16T08:39:55.150 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout pgs: 2024-09-16T08:39:55.150 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-09-16T08:39:55.150 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.524+0000 7f70e8082640 1 Processor -- start 2024-09-16T08:39:55.150 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.525+0000 7f70e8082640 1 -- start start 2024-09-16T08:39:55.150 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.525+0000 7f70e8082640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f70e0109aa0 con 0x7f70e0105620 2024-09-16T08:39:55.151 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.525+0000 7f70e5e1e640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f70e0105620 0x7f70e0105a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:37650/0 (socket says 172.21.15.160:37650) 2024-09-16T08:39:55.151 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.525+0000 7f70e5e1e640 1 -- 172.21.15.160:0/1920573299 learned_addr learned my addr 172.21.15.160:0/1920573299 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:39:55.151 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.525+0000 7f70e4e1c640 1 -- 172.21.15.160:0/1920573299 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1188391751 0 0) 0x7f70e0109aa0 con 0x7f70e0105620 2024-09-16T08:39:55.151 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.525+0000 7f70e4e1c640 1 -- 172.21.15.160:0/1920573299 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70cc003520 con 0x7f70e0105620 2024-09-16T08:39:55.151 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.526+0000 7f70e4e1c640 1 -- 172.21.15.160:0/1920573299 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2631683656 0 0) 0x7f70cc003520 con 0x7f70e0105620 2024-09-16T08:39:55.151 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.526+0000 7f70e4e1c640 1 -- 172.21.15.160:0/1920573299 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e010ac80 con 0x7f70e0105620 2024-09-16T08:39:55.151 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.526+0000 7f70e4e1c640 1 -- 172.21.15.160:0/1920573299 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f70d0001600 con 0x7f70e0105620 2024-09-16T08:39:55.151 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.526+0000 7f70e4e1c640 1 -- 172.21.15.160:0/1920573299 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f70d00018a0 con 0x7f70e0105620 2024-09-16T08:39:55.151 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.526+0000 7f70e4e1c640 1 -- 172.21.15.160:0/1920573299 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f70d0001ca0 con 0x7f70e0105620 2024-09-16T08:39:55.151 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.526+0000 7f70e8082640 1 -- 172.21.15.160:0/1920573299 >> v1:172.21.15.160:6789/0 conn(0x7f70e0105620 legacy=0x7f70e0105a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:39:55.151 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.526+0000 7f70e8082640 1 -- 172.21.15.160:0/1920573299 shutdown_connections 2024-09-16T08:39:55.151 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.526+0000 7f70e8082640 1 -- 172.21.15.160:0/1920573299 >> 172.21.15.160:0/1920573299 conn(0x7f70e0100f90 msgr2=0x7f70e01033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:39:55.152 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.526+0000 7f70e8082640 1 -- 172.21.15.160:0/1920573299 shutdown_connections 2024-09-16T08:39:55.152 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.526+0000 7f70e8082640 1 -- 172.21.15.160:0/1920573299 wait complete. 2024-09-16T08:39:55.152 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.527+0000 7f70e8082640 1 Processor -- start 2024-09-16T08:39:55.152 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.527+0000 7f70e8082640 1 -- start start 2024-09-16T08:39:55.152 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.527+0000 7f70e8082640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f70e019a2d0 con 0x7f70e0105620 2024-09-16T08:39:55.152 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.527+0000 7f70e5e1e640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f70e0105620 0x7f70e0199b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:37654/0 (socket says 172.21.15.160:37654) 2024-09-16T08:39:55.152 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.527+0000 7f70e5e1e640 1 -- 172.21.15.160:0/2344987462 learned_addr learned my addr 172.21.15.160:0/2344987462 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:39:55.152 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.527+0000 7f70d6ffd640 1 -- 172.21.15.160:0/2344987462 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3423955760 0 0) 0x7f70e019a2d0 con 0x7f70e0105620 2024-09-16T08:39:55.152 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.527+0000 7f70d6ffd640 1 -- 172.21.15.160:0/2344987462 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70cc003160 con 0x7f70e0105620 2024-09-16T08:39:55.152 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.528+0000 7f70d6ffd640 1 -- 172.21.15.160:0/2344987462 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 891771417 0 0) 0x7f70cc003160 con 0x7f70e0105620 2024-09-16T08:39:55.152 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.528+0000 7f70d6ffd640 1 -- 172.21.15.160:0/2344987462 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f70e019a2d0 con 0x7f70e0105620 2024-09-16T08:39:55.152 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.528+0000 7f70d6ffd640 1 -- 172.21.15.160:0/2344987462 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f70d0001b60 con 0x7f70e0105620 2024-09-16T08:39:55.153 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.528+0000 7f70d6ffd640 1 -- 172.21.15.160:0/2344987462 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2354279994 0 0) 0x7f70e019a2d0 con 0x7f70e0105620 2024-09-16T08:39:55.153 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.528+0000 7f70d6ffd640 1 -- 172.21.15.160:0/2344987462 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e019a4c0 con 0x7f70e0105620 2024-09-16T08:39:55.153 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.528+0000 7f70e8082640 1 -- 172.21.15.160:0/2344987462 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f70e019dda0 con 0x7f70e0105620 2024-09-16T08:39:55.153 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.528+0000 7f70e8082640 1 -- 172.21.15.160:0/2344987462 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f70e019e1c0 con 0x7f70e0105620 2024-09-16T08:39:55.153 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.528+0000 7f70d6ffd640 1 -- 172.21.15.160:0/2344987462 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f70d00018a0 con 0x7f70e0105620 2024-09-16T08:39:55.153 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.528+0000 7f70d6ffd640 1 -- 172.21.15.160:0/2344987462 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f70d0004d40 con 0x7f70e0105620 2024-09-16T08:39:55.153 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.528+0000 7f70d6ffd640 1 -- 172.21.15.160:0/2344987462 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f70d00052f0 con 0x7f70e0105620 2024-09-16T08:39:55.153 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.528+0000 7f70d6ffd640 1 -- 172.21.15.160:0/2344987462 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3090976605 0 0) 0x7f70d0006a20 con 0x7f70e0105620 2024-09-16T08:39:55.153 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.529+0000 7f70e8082640 1 -- 172.21.15.160:0/2344987462 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c40050b0 con 0x7f70e0105620 2024-09-16T08:39:55.153 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.531+0000 7f70d6ffd640 1 -- 172.21.15.160:0/2344987462 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (unknown 1092875540 0 1702466736) 0x7f70d00057c0 con 0x7f70e0105620 2024-09-16T08:39:55.153 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.592+0000 7f70e8082640 1 -- 172.21.15.160:0/2344987462 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "status"} v 0) -- 0x7f70c4005bb0 con 0x7f70e0105620 2024-09-16T08:39:55.153 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.592+0000 7f70d6ffd640 1 -- 172.21.15.160:0/2344987462 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (unknown 1155462804 0 1012106234) 0x7f70d00059a0 con 0x7f70e0105620 2024-09-16T08:39:55.154 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.593+0000 7f70e8082640 1 -- 172.21.15.160:0/2344987462 >> v1:172.21.15.160:6789/0 conn(0x7f70e0105620 legacy=0x7f70e0199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:39:55.154 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.593+0000 7f70e8082640 1 -- 172.21.15.160:0/2344987462 shutdown_connections 2024-09-16T08:39:55.154 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.593+0000 7f70e8082640 1 -- 172.21.15.160:0/2344987462 >> 172.21.15.160:0/2344987462 conn(0x7f70e0100f90 msgr2=0x7f70e007a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:39:55.154 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.594+0000 7f70e8082640 1 -- 172.21.15.160:0/2344987462 shutdown_connections 2024-09-16T08:39:55.154 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:54.594+0000 7f70e8082640 1 -- 172.21.15.160:0/2344987462 wait complete. 2024-09-16T08:39:55.154 INFO:teuthology.orchestra.run.smithi160.stdout:mon is available 2024-09-16T08:39:55.154 INFO:teuthology.orchestra.run.smithi160.stdout:Assimilating anything we can from ceph.conf... 2024-09-16T08:39:55.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:55 smithi160 ceph-mon[29668]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T08:39:55.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:55 smithi160 ceph-mon[29668]: monmap epoch 1 2024-09-16T08:39:55.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:55 smithi160 ceph-mon[29668]: fsid 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:39:55.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:55 smithi160 ceph-mon[29668]: last_changed 2024-09-16T08:39:50.898512+0000 2024-09-16T08:39:55.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:55 smithi160 ceph-mon[29668]: created 2024-09-16T08:39:50.898512+0000 2024-09-16T08:39:55.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:55 smithi160 ceph-mon[29668]: min_mon_release 19 (squid) 2024-09-16T08:39:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:55 smithi160 ceph-mon[29668]: election_strategy: 1 2024-09-16T08:39:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:55 smithi160 ceph-mon[29668]: 0: v1:172.21.15.160:6789/0 mon.a 2024-09-16T08:39:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:55 smithi160 ceph-mon[29668]: fsmap 2024-09-16T08:39:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:55 smithi160 ceph-mon[29668]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T08:39:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:55 smithi160 ceph-mon[29668]: mgrmap e1: no daemons active 2024-09-16T08:39:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:55 smithi160 ceph-mon[29668]: from='client.? v1:172.21.15.160:0/2344987462' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-16T08:39:56.289 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:56 smithi160 ceph-mon[29668]: from='client.? v1:172.21.15.160:0/3465334338' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-16T08:39:56.289 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:56 smithi160 ceph-mon[29668]: from='client.? v1:172.21.15.160:0/3465334338' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T08:39:56.298 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-09-16T08:39:56.298 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T08:39:56.298 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout fsid = 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:39:56.298 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-16T08:39:56.298 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_host = [v1:172.21.15.160:6789] 2024-09-16T08:39:56.298 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T08:39:56.298 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T08:39:56.298 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T08:39:56.298 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T08:39:56.298 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-09-16T08:39:56.298 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T08:39:56.298 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T08:39:56.299 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-09-16T08:39:56.299 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T08:39:56.299 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T08:39:56.299 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T08:39:56.299 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.605+0000 7f1c86594640 1 Processor -- start 2024-09-16T08:39:56.299 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.605+0000 7f1c86594640 1 -- start start 2024-09-16T08:39:56.299 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.605+0000 7f1c86594640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1c8010aba0 con 0x7f1c80106720 2024-09-16T08:39:56.299 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.606+0000 7f1c7ffff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f1c80106720 0x7f1c80106b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:37656/0 (socket says 172.21.15.160:37656) 2024-09-16T08:39:56.299 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.606+0000 7f1c7ffff640 1 -- 172.21.15.160:0/916021772 learned_addr learned my addr 172.21.15.160:0/916021772 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:39:56.299 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.606+0000 7f1c7effd640 1 -- 172.21.15.160:0/916021772 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2927907138 0 0) 0x7f1c8010aba0 con 0x7f1c80106720 2024-09-16T08:39:56.299 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.606+0000 7f1c7effd640 1 -- 172.21.15.160:0/916021772 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c68003520 con 0x7f1c80106720 2024-09-16T08:39:56.299 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.606+0000 7f1c7effd640 1 -- 172.21.15.160:0/916021772 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1584503431 0 0) 0x7f1c68003520 con 0x7f1c80106720 2024-09-16T08:39:56.299 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.606+0000 7f1c7effd640 1 -- 172.21.15.160:0/916021772 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8010bd80 con 0x7f1c80106720 2024-09-16T08:39:56.300 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.606+0000 7f1c7effd640 1 -- 172.21.15.160:0/916021772 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f1c70001600 con 0x7f1c80106720 2024-09-16T08:39:56.300 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.606+0000 7f1c7effd640 1 -- 172.21.15.160:0/916021772 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1c700018a0 con 0x7f1c80106720 2024-09-16T08:39:56.300 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.606+0000 7f1c7effd640 1 -- 172.21.15.160:0/916021772 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f1c70001ca0 con 0x7f1c80106720 2024-09-16T08:39:56.300 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.606+0000 7f1c86594640 1 -- 172.21.15.160:0/916021772 >> v1:172.21.15.160:6789/0 conn(0x7f1c80106720 legacy=0x7f1c80106b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:39:56.300 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.607+0000 7f1c86594640 1 -- 172.21.15.160:0/916021772 shutdown_connections 2024-09-16T08:39:56.300 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.607+0000 7f1c86594640 1 -- 172.21.15.160:0/916021772 >> 172.21.15.160:0/916021772 conn(0x7f1c80102070 msgr2=0x7f1c80104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:39:56.300 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.607+0000 7f1c86594640 1 -- 172.21.15.160:0/916021772 shutdown_connections 2024-09-16T08:39:56.300 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.607+0000 7f1c86594640 1 -- 172.21.15.160:0/916021772 wait complete. 2024-09-16T08:39:56.300 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.607+0000 7f1c86594640 1 Processor -- start 2024-09-16T08:39:56.300 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.607+0000 7f1c86594640 1 -- start start 2024-09-16T08:39:56.300 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.607+0000 7f1c86594640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1c8007c8d0 con 0x7f1c80106720 2024-09-16T08:39:56.300 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.608+0000 7f1c7ffff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f1c80106720 0x7f1c8007c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:37662/0 (socket says 172.21.15.160:37662) 2024-09-16T08:39:56.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.608+0000 7f1c7ffff640 1 -- 172.21.15.160:0/3465334338 learned_addr learned my addr 172.21.15.160:0/3465334338 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:39:56.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.608+0000 7f1c7d7fa640 1 -- 172.21.15.160:0/3465334338 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1492704231 0 0) 0x7f1c8007c8d0 con 0x7f1c80106720 2024-09-16T08:39:56.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.608+0000 7f1c7d7fa640 1 -- 172.21.15.160:0/3465334338 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c68003160 con 0x7f1c80106720 2024-09-16T08:39:56.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.608+0000 7f1c7d7fa640 1 -- 172.21.15.160:0/3465334338 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3966439228 0 0) 0x7f1c68003160 con 0x7f1c80106720 2024-09-16T08:39:56.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.608+0000 7f1c7d7fa640 1 -- 172.21.15.160:0/3465334338 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1c8007c8d0 con 0x7f1c80106720 2024-09-16T08:39:56.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.608+0000 7f1c7d7fa640 1 -- 172.21.15.160:0/3465334338 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f1c70001b60 con 0x7f1c80106720 2024-09-16T08:39:56.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.608+0000 7f1c7d7fa640 1 -- 172.21.15.160:0/3465334338 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3483117241 0 0) 0x7f1c8007c8d0 con 0x7f1c80106720 2024-09-16T08:39:56.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.608+0000 7f1c7d7fa640 1 -- 172.21.15.160:0/3465334338 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c801a0bb0 con 0x7f1c80106720 2024-09-16T08:39:56.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.608+0000 7f1c86594640 1 -- 172.21.15.160:0/3465334338 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1c8019fc00 con 0x7f1c80106720 2024-09-16T08:39:56.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.608+0000 7f1c86594640 1 -- 172.21.15.160:0/3465334338 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f1c801a0140 con 0x7f1c80106720 2024-09-16T08:39:56.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.608+0000 7f1c7d7fa640 1 -- 172.21.15.160:0/3465334338 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1c700018a0 con 0x7f1c80106720 2024-09-16T08:39:56.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.608+0000 7f1c7d7fa640 1 -- 172.21.15.160:0/3465334338 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f1c70004d00 con 0x7f1c80106720 2024-09-16T08:39:56.301 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.608+0000 7f1c7d7fa640 1 -- 172.21.15.160:0/3465334338 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f1c700052b0 con 0x7f1c80106720 2024-09-16T08:39:56.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.608+0000 7f1c7d7fa640 1 -- 172.21.15.160:0/3465334338 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3090976605 0 0) 0x7f1c70006810 con 0x7f1c80106720 2024-09-16T08:39:56.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.609+0000 7f1c86594640 1 -- 172.21.15.160:0/3465334338 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c6c0050b0 con 0x7f1c80106720 2024-09-16T08:39:56.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.611+0000 7f1c7d7fa640 1 -- 172.21.15.160:0/3465334338 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (unknown 1092875540 0 1702466736) 0x7f1c70006b80 con 0x7f1c80106720 2024-09-16T08:39:56.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.670+0000 7f1c86594640 1 -- 172.21.15.160:0/3465334338 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f1c6c002420 con 0x7f1c80106720 2024-09-16T08:39:56.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.673+0000 7f1c7d7fa640 1 -- 172.21.15.160:0/3465334338 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+354 (unknown 1213389831 0 4075966108) 0x7f1c700058f0 con 0x7f1c80106720 2024-09-16T08:39:56.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.673+0000 7f1c7d7fa640 1 -- 172.21.15.160:0/3465334338 <== mon.0 v1:172.21.15.160:6789/0 11 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f1c70005f20 con 0x7f1c80106720 2024-09-16T08:39:56.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.674+0000 7f1c86594640 1 -- 172.21.15.160:0/3465334338 >> v1:172.21.15.160:6789/0 conn(0x7f1c80106720 legacy=0x7f1c8007c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:39:56.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.674+0000 7f1c86594640 1 -- 172.21.15.160:0/3465334338 shutdown_connections 2024-09-16T08:39:56.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.674+0000 7f1c86594640 1 -- 172.21.15.160:0/3465334338 >> 172.21.15.160:0/3465334338 conn(0x7f1c80102070 msgr2=0x7f1c80190c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:39:56.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.674+0000 7f1c86594640 1 -- 172.21.15.160:0/3465334338 shutdown_connections 2024-09-16T08:39:56.302 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:55.674+0000 7f1c86594640 1 -- 172.21.15.160:0/3465334338 wait complete. 2024-09-16T08:39:56.302 INFO:teuthology.orchestra.run.smithi160.stdout:Generating new minimal ceph.conf... 2024-09-16T08:39:57.394 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:57 smithi160 ceph-mon[29668]: from='client.? v1:172.21.15.160:0/4165459498' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:39:57.404 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.728+0000 7f73c6034640 1 Processor -- start 2024-09-16T08:39:57.404 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.728+0000 7f73c6034640 1 -- start start 2024-09-16T08:39:57.404 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.728+0000 7f73c6034640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f73c010ab90 con 0x7f73c0106710 2024-09-16T08:39:57.404 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.728+0000 7f73bf7fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f73c0106710 0x7f73c0106b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:37678/0 (socket says 172.21.15.160:37678) 2024-09-16T08:39:57.404 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.728+0000 7f73bf7fe640 1 -- 172.21.15.160:0/978931676 learned_addr learned my addr 172.21.15.160:0/978931676 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:39:57.405 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.729+0000 7f73be7fc640 1 -- 172.21.15.160:0/978931676 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 595587768 0 0) 0x7f73c010ab90 con 0x7f73c0106710 2024-09-16T08:39:57.405 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.729+0000 7f73be7fc640 1 -- 172.21.15.160:0/978931676 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73a8003520 con 0x7f73c0106710 2024-09-16T08:39:57.405 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.729+0000 7f73be7fc640 1 -- 172.21.15.160:0/978931676 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2739926440 0 0) 0x7f73a8003520 con 0x7f73c0106710 2024-09-16T08:39:57.405 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.729+0000 7f73be7fc640 1 -- 172.21.15.160:0/978931676 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c010bd70 con 0x7f73c0106710 2024-09-16T08:39:57.405 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.729+0000 7f73be7fc640 1 -- 172.21.15.160:0/978931676 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f73b0001600 con 0x7f73c0106710 2024-09-16T08:39:57.405 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.729+0000 7f73be7fc640 1 -- 172.21.15.160:0/978931676 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f73b0001cf0 con 0x7f73c0106710 2024-09-16T08:39:57.405 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.729+0000 7f73be7fc640 1 -- 172.21.15.160:0/978931676 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f73b00058d0 con 0x7f73c0106710 2024-09-16T08:39:57.406 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.729+0000 7f73c6034640 1 -- 172.21.15.160:0/978931676 >> v1:172.21.15.160:6789/0 conn(0x7f73c0106710 legacy=0x7f73c0106b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:39:57.406 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.729+0000 7f73c6034640 1 -- 172.21.15.160:0/978931676 shutdown_connections 2024-09-16T08:39:57.406 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.729+0000 7f73c6034640 1 -- 172.21.15.160:0/978931676 >> 172.21.15.160:0/978931676 conn(0x7f73c0102000 msgr2=0x7f73c0104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:39:57.406 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.730+0000 7f73c6034640 1 -- 172.21.15.160:0/978931676 shutdown_connections 2024-09-16T08:39:57.406 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.730+0000 7f73c6034640 1 -- 172.21.15.160:0/978931676 wait complete. 2024-09-16T08:39:57.406 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.730+0000 7f73c6034640 1 Processor -- start 2024-09-16T08:39:57.406 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.730+0000 7f73c6034640 1 -- start start 2024-09-16T08:39:57.406 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.730+0000 7f73c6034640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f73c019aa30 con 0x7f73c0106710 2024-09-16T08:39:57.407 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.730+0000 7f73bf7fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f73c0106710 0x7f73c019a2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:37694/0 (socket says 172.21.15.160:37694) 2024-09-16T08:39:57.407 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.730+0000 7f73bf7fe640 1 -- 172.21.15.160:0/4165459498 learned_addr learned my addr 172.21.15.160:0/4165459498 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:39:57.407 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.731+0000 7f73bcff9640 1 -- 172.21.15.160:0/4165459498 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3155115545 0 0) 0x7f73c019aa30 con 0x7f73c0106710 2024-09-16T08:39:57.407 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.731+0000 7f73bcff9640 1 -- 172.21.15.160:0/4165459498 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73a8005570 con 0x7f73c0106710 2024-09-16T08:39:57.407 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.731+0000 7f73bcff9640 1 -- 172.21.15.160:0/4165459498 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 377859850 0 0) 0x7f73a8005570 con 0x7f73c0106710 2024-09-16T08:39:57.407 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.731+0000 7f73bcff9640 1 -- 172.21.15.160:0/4165459498 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f73c019aa30 con 0x7f73c0106710 2024-09-16T08:39:57.407 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.731+0000 7f73bcff9640 1 -- 172.21.15.160:0/4165459498 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f73b0001080 con 0x7f73c0106710 2024-09-16T08:39:57.407 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.731+0000 7f73bcff9640 1 -- 172.21.15.160:0/4165459498 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 465093138 0 0) 0x7f73c019aa30 con 0x7f73c0106710 2024-09-16T08:39:57.407 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.731+0000 7f73bcff9640 1 -- 172.21.15.160:0/4165459498 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c019ac20 con 0x7f73c0106710 2024-09-16T08:39:57.407 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.731+0000 7f73c6034640 1 -- 172.21.15.160:0/4165459498 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f73c019e500 con 0x7f73c0106710 2024-09-16T08:39:57.407 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.731+0000 7f73c6034640 1 -- 172.21.15.160:0/4165459498 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f73c019e920 con 0x7f73c0106710 2024-09-16T08:39:57.407 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.731+0000 7f73bcff9640 1 -- 172.21.15.160:0/4165459498 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f73b0001820 con 0x7f73c0106710 2024-09-16T08:39:57.407 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.731+0000 7f73bcff9640 1 -- 172.21.15.160:0/4165459498 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f73b0006030 con 0x7f73c0106710 2024-09-16T08:39:57.408 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.731+0000 7f73bcff9640 1 -- 172.21.15.160:0/4165459498 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f73b0006600 con 0x7f73c0106710 2024-09-16T08:39:57.408 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.731+0000 7f73bcff9640 1 -- 172.21.15.160:0/4165459498 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3090976605 0 0) 0x7f73b0007c70 con 0x7f73c0106710 2024-09-16T08:39:57.408 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.732+0000 7f73c6034640 1 -- 172.21.15.160:0/4165459498 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ac0050b0 con 0x7f73c0106710 2024-09-16T08:39:57.408 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.734+0000 7f73bcff9640 1 -- 172.21.15.160:0/4165459498 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (unknown 1092875540 0 1702466736) 0x7f73b0001cb0 con 0x7f73c0106710 2024-09-16T08:39:57.408 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.793+0000 7f73c6034640 1 -- 172.21.15.160:0/4165459498 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f73ac0053a0 con 0x7f73c0106710 2024-09-16T08:39:57.408 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.793+0000 7f73bcff9640 1 -- 172.21.15.160:0/4165459498 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+148 (unknown 1452402520 0 157508626) 0x7f73b00049b0 con 0x7f73c0106710 2024-09-16T08:39:57.408 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.794+0000 7f73c6034640 1 -- 172.21.15.160:0/4165459498 >> v1:172.21.15.160:6789/0 conn(0x7f73c0106710 legacy=0x7f73c019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:39:57.408 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.794+0000 7f73c6034640 1 -- 172.21.15.160:0/4165459498 shutdown_connections 2024-09-16T08:39:57.408 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.794+0000 7f73c6034640 1 -- 172.21.15.160:0/4165459498 >> 172.21.15.160:0/4165459498 conn(0x7f73c0102000 msgr2=0x7f73c01043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:39:57.408 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.794+0000 7f73c6034640 1 -- 172.21.15.160:0/4165459498 shutdown_connections 2024-09-16T08:39:57.409 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:56.794+0000 7f73c6034640 1 -- 172.21.15.160:0/4165459498 wait complete. 2024-09-16T08:39:57.409 INFO:teuthology.orchestra.run.smithi160.stdout:Restarting the monitor... 2024-09-16T08:39:57.675 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:57 smithi160 systemd[1]: Stopping Ceph mon.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T08:39:57.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:57 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-a[29645]: 2024-09-16T08:39:57.673+0000 7f2724813640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:39:57.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:57 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-a[29645]: 2024-09-16T08:39:57.673+0000 7f2724813640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-16T08:39:57.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:57 smithi160 podman[29983]: 2024-09-16 08:39:57.848914555 +0000 UTC m=+0.287553857 container died 511bef551290098faf2bbe56777833282856ee5d69cb03cb04be29645871ff5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True) 2024-09-16T08:39:58.349 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:58 smithi160 podman[29983]: 2024-09-16 08:39:58.172315898 +0000 UTC m=+0.610955200 container cleanup 511bef551290098faf2bbe56777833282856ee5d69cb03cb04be29645871ff5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T08:39:58.349 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:58 smithi160 bash[29983]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-a 2024-09-16T08:39:58.349 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:58 smithi160 podman[30000]: 2024-09-16 08:39:58.34817347 +0000 UTC m=+0.498780069 container remove 511bef551290098faf2bbe56777833282856ee5d69cb03cb04be29645871ff5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-16T08:39:58.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:58 smithi160 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-16T08:39:58.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:58 smithi160 systemd[1]: Stopped Ceph mon.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:39:58.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:58 smithi160 systemd[1]: Starting Ceph mon.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T08:39:59.068 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:58 smithi160 podman[30100]: 2024-09-16 08:39:58.782364285 +0000 UTC m=+0.100998018 container create a8429aef53bc2ece1ba7d93da44da43b19a5715ead119fdc0a681a6d0110d64a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-16T08:39:59.068 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:58 smithi160 podman[30100]: 2024-09-16 08:39:58.696855848 +0000 UTC m=+0.015489578 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:39:59.068 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:58 smithi160 podman[30100]: 2024-09-16 08:39:58.976211081 +0000 UTC m=+0.294844811 container init a8429aef53bc2ece1ba7d93da44da43b19a5715ead119fdc0a681a6d0110d64a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T08:39:59.068 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:58 smithi160 podman[30100]: 2024-09-16 08:39:58.98107731 +0000 UTC m=+0.299711041 container start a8429aef53bc2ece1ba7d93da44da43b19a5715ead119fdc0a681a6d0110d64a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2) 2024-09-16T08:39:59.068 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T08:39:59.068 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev), process ceph-mon, pid 7 2024-09-16T08:39:59.068 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: pidfile_write: ignore empty --pid-file 2024-09-16T08:39:59.068 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: load: jerasure load: lrc 2024-09-16T08:39:59.068 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: RocksDB version: 7.9.2 2024-09-16T08:39:59.069 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Git sha 0 2024-09-16T08:39:59.069 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Compile date 2024-09-13 14:50:41 2024-09-16T08:39:59.069 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: DB SUMMARY 2024-09-16T08:39:59.069 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: DB Session ID: HTXGV73VNCTPKNTXPHKA 2024-09-16T08:39:59.069 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: CURRENT file: CURRENT 2024-09-16T08:39:59.069 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T08:39:59.069 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-16T08:39:59.069 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-16T08:39:59.069 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 88916 ; 2024-09-16T08:39:59.069 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.error_if_exists: 0 2024-09-16T08:39:59.069 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.create_if_missing: 0 2024-09-16T08:39:59.069 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.paranoid_checks: 1 2024-09-16T08:39:59.070 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T08:39:59.070 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T08:39:59.070 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T08:39:59.070 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.env: 0x558003c48120 2024-09-16T08:39:59.070 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T08:39:59.070 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.info_log: 0x5580052716a0 2024-09-16T08:39:59.070 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T08:39:59.070 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.statistics: (nil) 2024-09-16T08:39:59.070 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.use_fsync: 0 2024-09-16T08:39:59.070 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_log_file_size: 0 2024-09-16T08:39:59.070 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T08:39:59.071 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T08:39:59.071 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T08:39:59.071 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T08:39:59.071 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.allow_fallocate: 1 2024-09-16T08:39:59.071 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T08:39:59.071 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T08:39:59.071 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.use_direct_reads: 0 2024-09-16T08:39:59.071 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T08:39:59.071 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T08:39:59.071 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.db_log_dir: 2024-09-16T08:39:59.071 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.wal_dir: 2024-09-16T08:39:59.072 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T08:39:59.072 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T08:39:59.072 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T08:39:59.072 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T08:39:59.072 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T08:39:59.072 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T08:39:59.072 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T08:39:59.072 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T08:39:59.072 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.write_buffer_manager: 0x558005275b80 2024-09-16T08:39:59.072 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T08:39:59.072 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T08:39:59.072 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T08:39:59.072 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T08:39:59.073 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T08:39:59.074 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T08:39:59.074 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T08:39:59.074 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T08:39:59.074 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.unordered_write: 0 2024-09-16T08:39:59.074 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T08:39:59.074 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T08:39:59.074 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T08:39:59.074 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T08:39:59.075 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.row_cache: None 2024-09-16T08:39:59.075 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.wal_filter: None 2024-09-16T08:39:59.075 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T08:39:59.075 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T08:39:59.075 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.two_write_queues: 0 2024-09-16T08:39:59.075 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T08:39:59.075 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.wal_compression: 0 2024-09-16T08:39:59.075 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.atomic_flush: 0 2024-09-16T08:39:59.075 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T08:39:59.075 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T08:39:59.075 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T08:39:59.075 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.log_readahead_size: 0 2024-09-16T08:39:59.076 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T08:39:59.076 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T08:39:59.076 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T08:39:59.076 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T08:39:59.076 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T08:39:59.076 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T08:39:59.076 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T08:39:59.076 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_background_jobs: 2 2024-09-16T08:39:59.076 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_background_compactions: -1 2024-09-16T08:39:59.076 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_subcompactions: 1 2024-09-16T08:39:59.076 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T08:39:59.076 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T08:39:59.076 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T08:39:59.077 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T08:39:59.077 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T08:39:59.077 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T08:39:59.077 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T08:39:59.077 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T08:39:59.077 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_open_files: -1 2024-09-16T08:39:59.077 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T08:39:59.077 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T08:39:59.077 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T08:39:59.077 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T08:39:59.077 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_background_flushes: -1 2024-09-16T08:39:59.077 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Compression algorithms supported: 2024-09-16T08:39:59.077 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: kZSTD supported: 0 2024-09-16T08:39:59.078 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: kXpressCompression supported: 0 2024-09-16T08:39:59.078 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: kBZip2Compression supported: 0 2024-09-16T08:39:59.078 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T08:39:59.078 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: kLZ4Compression supported: 1 2024-09-16T08:39:59.078 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: kZlibCompression supported: 1 2024-09-16T08:39:59.078 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T08:39:59.078 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: kSnappyCompression supported: 1 2024-09-16T08:39:59.078 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T08:39:59.078 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T08:39:59.078 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-16T08:39:59.078 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T08:39:59.079 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T08:39:59.079 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.merge_operator: 2024-09-16T08:39:59.079 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compaction_filter: None 2024-09-16T08:39:59.079 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T08:39:59.079 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T08:39:59.079 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T08:39:59.079 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T08:39:59.080 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5580052712c0) 2024-09-16T08:39:59.080 INFO:journalctl@ceph.mon.a.smithi160.stdout: cache_index_and_filter_blocks: 1 2024-09-16T08:39:59.080 INFO:journalctl@ceph.mon.a.smithi160.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T08:39:59.080 INFO:journalctl@ceph.mon.a.smithi160.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T08:39:59.080 INFO:journalctl@ceph.mon.a.smithi160.stdout: pin_top_level_index_and_filter: 1 2024-09-16T08:39:59.080 INFO:journalctl@ceph.mon.a.smithi160.stdout: index_type: 0 2024-09-16T08:39:59.080 INFO:journalctl@ceph.mon.a.smithi160.stdout: data_block_index_type: 0 2024-09-16T08:39:59.080 INFO:journalctl@ceph.mon.a.smithi160.stdout: index_shortening: 1 2024-09-16T08:39:59.081 INFO:journalctl@ceph.mon.a.smithi160.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T08:39:59.081 INFO:journalctl@ceph.mon.a.smithi160.stdout: checksum: 4 2024-09-16T08:39:59.081 INFO:journalctl@ceph.mon.a.smithi160.stdout: no_block_cache: 0 2024-09-16T08:39:59.081 INFO:journalctl@ceph.mon.a.smithi160.stdout: block_cache: 0x558005265610 2024-09-16T08:39:59.081 INFO:journalctl@ceph.mon.a.smithi160.stdout: block_cache_name: BinnedLRUCache 2024-09-16T08:39:59.081 INFO:journalctl@ceph.mon.a.smithi160.stdout: block_cache_options: 2024-09-16T08:39:59.081 INFO:journalctl@ceph.mon.a.smithi160.stdout: capacity : 536870912 2024-09-16T08:39:59.081 INFO:journalctl@ceph.mon.a.smithi160.stdout: num_shard_bits : 4 2024-09-16T08:39:59.081 INFO:journalctl@ceph.mon.a.smithi160.stdout: strict_capacity_limit : 0 2024-09-16T08:39:59.081 INFO:journalctl@ceph.mon.a.smithi160.stdout: high_pri_pool_ratio: 0.000 2024-09-16T08:39:59.081 INFO:journalctl@ceph.mon.a.smithi160.stdout: block_cache_compressed: (nil) 2024-09-16T08:39:59.081 INFO:journalctl@ceph.mon.a.smithi160.stdout: persistent_cache: (nil) 2024-09-16T08:39:59.081 INFO:journalctl@ceph.mon.a.smithi160.stdout: block_size: 4096 2024-09-16T08:39:59.082 INFO:journalctl@ceph.mon.a.smithi160.stdout: block_size_deviation: 10 2024-09-16T08:39:59.082 INFO:journalctl@ceph.mon.a.smithi160.stdout: block_restart_interval: 16 2024-09-16T08:39:59.082 INFO:journalctl@ceph.mon.a.smithi160.stdout: index_block_restart_interval: 1 2024-09-16T08:39:59.082 INFO:journalctl@ceph.mon.a.smithi160.stdout: metadata_block_size: 4096 2024-09-16T08:39:59.082 INFO:journalctl@ceph.mon.a.smithi160.stdout: partition_filters: 0 2024-09-16T08:39:59.082 INFO:journalctl@ceph.mon.a.smithi160.stdout: use_delta_encoding: 1 2024-09-16T08:39:59.082 INFO:journalctl@ceph.mon.a.smithi160.stdout: filter_policy: bloomfilter 2024-09-16T08:39:59.082 INFO:journalctl@ceph.mon.a.smithi160.stdout: whole_key_filtering: 1 2024-09-16T08:39:59.082 INFO:journalctl@ceph.mon.a.smithi160.stdout: verify_compression: 0 2024-09-16T08:39:59.082 INFO:journalctl@ceph.mon.a.smithi160.stdout: read_amp_bytes_per_bit: 0 2024-09-16T08:39:59.082 INFO:journalctl@ceph.mon.a.smithi160.stdout: format_version: 5 2024-09-16T08:39:59.082 INFO:journalctl@ceph.mon.a.smithi160.stdout: enable_index_compression: 1 2024-09-16T08:39:59.082 INFO:journalctl@ceph.mon.a.smithi160.stdout: block_align: 0 2024-09-16T08:39:59.083 INFO:journalctl@ceph.mon.a.smithi160.stdout: max_auto_readahead_size: 262144 2024-09-16T08:39:59.083 INFO:journalctl@ceph.mon.a.smithi160.stdout: prepopulate_block_cache: 0 2024-09-16T08:39:59.083 INFO:journalctl@ceph.mon.a.smithi160.stdout: initial_auto_readahead_size: 8192 2024-09-16T08:39:59.083 INFO:journalctl@ceph.mon.a.smithi160.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T08:39:59.083 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T08:39:59.083 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T08:39:59.083 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compression: NoCompression 2024-09-16T08:39:59.083 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T08:39:59.083 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T08:39:59.083 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T08:39:59.083 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.num_levels: 7 2024-09-16T08:39:59.083 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T08:39:59.083 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T08:39:59.083 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T08:39:59.084 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T08:39:59.084 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T08:39:59.084 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T08:39:59.084 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T08:39:59.084 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:39:59.084 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T08:39:59.084 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T08:39:59.084 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:39:59.084 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:39:59.084 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T08:39:59.084 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T08:39:59.084 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T08:39:59.084 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T08:39:59.085 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:39:59.085 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:39:59.085 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T08:39:59.085 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T08:39:59.085 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:39:59.085 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T08:39:59.085 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T08:39:59.085 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T08:39:59.085 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T08:39:59.085 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T08:39:59.085 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T08:39:59.085 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T08:39:59.085 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T08:39:59.086 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T08:39:59.086 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T08:39:59.086 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T08:39:59.086 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T08:39:59.086 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T08:39:59.086 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T08:39:59.086 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T08:39:59.086 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T08:39:59.086 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T08:39:59.086 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T08:39:59.086 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T08:39:59.086 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T08:39:59.086 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T08:39:59.086 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T08:39:59.087 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T08:39:59.087 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T08:39:59.087 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T08:39:59.087 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T08:39:59.087 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T08:39:59.087 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T08:39:59.087 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T08:39:59.087 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T08:39:59.087 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T08:39:59.087 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T08:39:59.087 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T08:39:59.087 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.inplace_update_support: 0 2024-09-16T08:39:59.087 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T08:39:59.087 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T08:39:59.088 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T08:39:59.088 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T08:39:59.088 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.bloom_locality: 0 2024-09-16T08:39:59.088 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.max_successive_merges: 0 2024-09-16T08:39:59.088 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T08:39:59.088 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T08:39:59.088 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T08:39:59.088 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T08:39:59.088 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.ttl: 2592000 2024-09-16T08:39:59.088 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T08:39:59.088 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T08:39:59.088 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T08:39:59.089 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.enable_blob_files: false 2024-09-16T08:39:59.089 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.min_blob_size: 0 2024-09-16T08:39:59.089 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T08:39:59.089 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T08:39:59.089 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T08:39:59.089 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T08:39:59.089 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T08:39:59.089 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T08:39:59.089 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T08:39:59.089 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T08:39:59.089 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-16T08:39:59.089 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-16T08:39:59.089 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bbf55989-271d-4039-b19b-58c8e5e56765 2024-09-16T08:39:59.090 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726475999008335, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-16T08:39:59.090 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-16T08:39:59.090 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726475999009233, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 85329, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 271, "table_properties": {"data_size": 83475, "index_size": 238, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10821, "raw_average_key_size": 47, "raw_value_size": 77338, "raw_average_value_size": 336, "num_data_blocks": 10, "num_entries": 230, "num_filter_entries": 230, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726475999, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bbf55989-271d-4039-b19b-58c8e5e56765", "db_session_id": "HTXGV73VNCTPKNTXPHKA", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-16T08:39:59.090 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726475999009329, "job": 1, "event": "recovery_finished"} 2024-09-16T08:39:59.090 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-16T08:39:59.090 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T08:39:59.090 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558005294e00 2024-09-16T08:39:59.090 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: DB pointer 0x558005396000 2024-09-16T08:39:59.090 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T08:39:59.090 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T08:39:59.090 INFO:journalctl@ceph.mon.a.smithi160.stdout: ** DB Stats ** 2024-09-16T08:39:59.090 INFO:journalctl@ceph.mon.a.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:39:59.090 INFO:journalctl@ceph.mon.a.smithi160.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T08:39:59.091 INFO:journalctl@ceph.mon.a.smithi160.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:39:59.091 INFO:journalctl@ceph.mon.a.smithi160.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:39:59.091 INFO:journalctl@ceph.mon.a.smithi160.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T08:39:59.091 INFO:journalctl@ceph.mon.a.smithi160.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:39:59.091 INFO:journalctl@ceph.mon.a.smithi160.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:39:59.091 INFO:journalctl@ceph.mon.a.smithi160.stdout: 2024-09-16T08:39:59.091 INFO:journalctl@ceph.mon.a.smithi160.stdout: ** Compaction Stats [default] ** 2024-09-16T08:39:59.091 INFO:journalctl@ceph.mon.a.smithi160.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:39:59.091 INFO:journalctl@ceph.mon.a.smithi160.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T08:39:59.091 INFO:journalctl@ceph.mon.a.smithi160.stdout: L0 2/0 85.16 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T08:39:59.091 INFO:journalctl@ceph.mon.a.smithi160.stdout: Sum 2/0 85.16 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T08:39:59.091 INFO:journalctl@ceph.mon.a.smithi160.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T08:39:59.091 INFO:journalctl@ceph.mon.a.smithi160.stdout: 2024-09-16T08:39:59.091 INFO:journalctl@ceph.mon.a.smithi160.stdout: ** Compaction Stats [default] ** 2024-09-16T08:39:59.092 INFO:journalctl@ceph.mon.a.smithi160.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:39:59.092 INFO:journalctl@ceph.mon.a.smithi160.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T08:39:59.092 INFO:journalctl@ceph.mon.a.smithi160.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 141.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T08:39:59.092 INFO:journalctl@ceph.mon.a.smithi160.stdout: 2024-09-16T08:39:59.092 INFO:journalctl@ceph.mon.a.smithi160.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T08:39:59.092 INFO:journalctl@ceph.mon.a.smithi160.stdout: 2024-09-16T08:39:59.092 INFO:journalctl@ceph.mon.a.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:39:59.092 INFO:journalctl@ceph.mon.a.smithi160.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T08:39:59.092 INFO:journalctl@ceph.mon.a.smithi160.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T08:39:59.092 INFO:journalctl@ceph.mon.a.smithi160.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T08:39:59.092 INFO:journalctl@ceph.mon.a.smithi160.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T08:39:59.092 INFO:journalctl@ceph.mon.a.smithi160.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T08:39:59.093 INFO:journalctl@ceph.mon.a.smithi160.stdout: Cumulative compaction: 0.00 GB write, 26.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:39:59.093 INFO:journalctl@ceph.mon.a.smithi160.stdout: Interval compaction: 0.00 GB write, 26.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:39:59.093 INFO:journalctl@ceph.mon.a.smithi160.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T08:39:59.093 INFO:journalctl@ceph.mon.a.smithi160.stdout: Block cache BinnedLRUCache@0x558005265610#7 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-16T08:39:59.093 INFO:journalctl@ceph.mon.a.smithi160.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2024-09-16T08:39:59.093 INFO:journalctl@ceph.mon.a.smithi160.stdout: 2024-09-16T08:39:59.093 INFO:journalctl@ceph.mon.a.smithi160.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T08:39:59.093 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: starting mon.a rank 0 at public addrs v1:172.21.15.160:6789/0 at bind addrs v1:172.21.15.160:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:39:59.093 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: mon.a@-1(???) e1 preinit fsid 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:39:59.093 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: mon.a@-1(???).mds e1 new map 2024-09-16T08:39:59.093 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: mon.a@-1(???).mds e1 print_map 2024-09-16T08:39:59.093 INFO:journalctl@ceph.mon.a.smithi160.stdout: e1 2024-09-16T08:39:59.093 INFO:journalctl@ceph.mon.a.smithi160.stdout: btime 2024-09-16T08:39:54:060588+0000 2024-09-16T08:39:59.093 INFO:journalctl@ceph.mon.a.smithi160.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:39:59.094 INFO:journalctl@ceph.mon.a.smithi160.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T08:39:59.094 INFO:journalctl@ceph.mon.a.smithi160.stdout: legacy client fscid: -1 2024-09-16T08:39:59.094 INFO:journalctl@ceph.mon.a.smithi160.stdout: 2024-09-16T08:39:59.094 INFO:journalctl@ceph.mon.a.smithi160.stdout: No filesystems configured 2024-09-16T08:39:59.094 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T08:39:59.094 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:39:59.094 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:39:59.094 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:39:59.094 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 ceph-mon[30137]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-16T08:39:59.094 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 bash[30100]: a8429aef53bc2ece1ba7d93da44da43b19a5715ead119fdc0a681a6d0110d64a 2024-09-16T08:39:59.094 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:39:59 smithi160 systemd[1]: Started Ceph mon.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:39:59.095 INFO:teuthology.orchestra.run.smithi160.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-16T08:40:00.268 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:00 smithi160 ceph-mon[30137]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T08:40:00.269 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:00 smithi160 ceph-mon[30137]: monmap epoch 1 2024-09-16T08:40:00.269 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:00 smithi160 ceph-mon[30137]: fsid 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:40:00.269 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:00 smithi160 ceph-mon[30137]: last_changed 2024-09-16T08:39:50.898512+0000 2024-09-16T08:40:00.269 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:00 smithi160 ceph-mon[30137]: created 2024-09-16T08:39:50.898512+0000 2024-09-16T08:40:00.269 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:00 smithi160 ceph-mon[30137]: min_mon_release 19 (squid) 2024-09-16T08:40:00.269 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:00 smithi160 ceph-mon[30137]: election_strategy: 1 2024-09-16T08:40:00.269 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:00 smithi160 ceph-mon[30137]: 0: v1:172.21.15.160:6789/0 mon.a 2024-09-16T08:40:00.269 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:00 smithi160 ceph-mon[30137]: fsmap 2024-09-16T08:40:00.269 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:00 smithi160 ceph-mon[30137]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T08:40:00.269 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:00 smithi160 ceph-mon[30137]: mgrmap e1: no daemons active 2024-09-16T08:40:00.269 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/58208470' entity='client.admin' 2024-09-16T08:40:00.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.586+0000 7fd394b78640 1 Processor -- start 2024-09-16T08:40:00.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.586+0000 7fd394b78640 1 -- start start 2024-09-16T08:40:00.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.587+0000 7fd394b78640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd390109ad0 con 0x7fd390105650 2024-09-16T08:40:00.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.587+0000 7fd38e59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd390105650 0x7fd390105a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:39788/0 (socket says 172.21.15.160:39788) 2024-09-16T08:40:00.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.587+0000 7fd38e59c640 1 -- 172.21.15.160:0/3109358242 learned_addr learned my addr 172.21.15.160:0/3109358242 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:00.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.587+0000 7fd38d59a640 1 -- 172.21.15.160:0/3109358242 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 900284594 0 0) 0x7fd390109ad0 con 0x7fd390105650 2024-09-16T08:40:00.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.587+0000 7fd38d59a640 1 -- 172.21.15.160:0/3109358242 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd378003520 con 0x7fd390105650 2024-09-16T08:40:00.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.587+0000 7fd38d59a640 1 -- 172.21.15.160:0/3109358242 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1373098910 0 0) 0x7fd378003520 con 0x7fd390105650 2024-09-16T08:40:00.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.588+0000 7fd38d59a640 1 -- 172.21.15.160:0/3109358242 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39010acb0 con 0x7fd390105650 2024-09-16T08:40:00.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.588+0000 7fd38d59a640 1 -- 172.21.15.160:0/3109358242 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fd380001600 con 0x7fd390105650 2024-09-16T08:40:00.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.588+0000 7fd38d59a640 1 -- 172.21.15.160:0/3109358242 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7fd380001cf0 con 0x7fd390105650 2024-09-16T08:40:00.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.588+0000 7fd38d59a640 1 -- 172.21.15.160:0/3109358242 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fd3800058d0 con 0x7fd390105650 2024-09-16T08:40:00.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.588+0000 7fd394b78640 1 -- 172.21.15.160:0/3109358242 >> v1:172.21.15.160:6789/0 conn(0x7fd390105650 legacy=0x7fd390105a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:00.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.588+0000 7fd394b78640 1 -- 172.21.15.160:0/3109358242 shutdown_connections 2024-09-16T08:40:00.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.588+0000 7fd394b78640 1 -- 172.21.15.160:0/3109358242 >> 172.21.15.160:0/3109358242 conn(0x7fd390100f80 msgr2=0x7fd3901033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:00.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.588+0000 7fd394b78640 1 -- 172.21.15.160:0/3109358242 shutdown_connections 2024-09-16T08:40:00.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.588+0000 7fd394b78640 1 -- 172.21.15.160:0/3109358242 wait complete. 2024-09-16T08:40:00.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.589+0000 7fd394b78640 1 Processor -- start 2024-09-16T08:40:00.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.589+0000 7fd394b78640 1 -- start start 2024-09-16T08:40:00.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.589+0000 7fd394b78640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd39019e030 con 0x7fd390105650 2024-09-16T08:40:00.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.589+0000 7fd38e59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd390105650 0x7fd3900797c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:39800/0 (socket says 172.21.15.160:39800) 2024-09-16T08:40:00.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.589+0000 7fd38e59c640 1 -- 172.21.15.160:0/58208470 learned_addr learned my addr 172.21.15.160:0/58208470 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:00.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.589+0000 7fd37f7fe640 1 -- 172.21.15.160:0/58208470 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1348431576 0 0) 0x7fd39019e030 con 0x7fd390105650 2024-09-16T08:40:00.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.589+0000 7fd37f7fe640 1 -- 172.21.15.160:0/58208470 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd378005570 con 0x7fd390105650 2024-09-16T08:40:00.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.590+0000 7fd37f7fe640 1 -- 172.21.15.160:0/58208470 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1327233400 0 0) 0x7fd378005570 con 0x7fd390105650 2024-09-16T08:40:00.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.590+0000 7fd37f7fe640 1 -- 172.21.15.160:0/58208470 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd39019e030 con 0x7fd390105650 2024-09-16T08:40:00.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.590+0000 7fd37f7fe640 1 -- 172.21.15.160:0/58208470 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fd380001080 con 0x7fd390105650 2024-09-16T08:40:00.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.590+0000 7fd37f7fe640 1 -- 172.21.15.160:0/58208470 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2404015122 0 0) 0x7fd39019e030 con 0x7fd390105650 2024-09-16T08:40:00.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.590+0000 7fd37f7fe640 1 -- 172.21.15.160:0/58208470 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39019f210 con 0x7fd390105650 2024-09-16T08:40:00.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.590+0000 7fd394b78640 1 -- 172.21.15.160:0/58208470 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd39019e200 con 0x7fd390105650 2024-09-16T08:40:00.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.590+0000 7fd394b78640 1 -- 172.21.15.160:0/58208470 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd39019e790 con 0x7fd390105650 2024-09-16T08:40:00.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.590+0000 7fd37f7fe640 1 -- 172.21.15.160:0/58208470 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7fd380001820 con 0x7fd390105650 2024-09-16T08:40:00.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.590+0000 7fd37f7fe640 1 -- 172.21.15.160:0/58208470 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fd380006120 con 0x7fd390105650 2024-09-16T08:40:00.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.590+0000 7fd37f7fe640 1 -- 172.21.15.160:0/58208470 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7fd380006650 con 0x7fd390105650 2024-09-16T08:40:00.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.590+0000 7fd37f7fe640 1 -- 172.21.15.160:0/58208470 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3090976605 0 0) 0x7fd380007c50 con 0x7fd390105650 2024-09-16T08:40:00.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.591+0000 7fd394b78640 1 -- 172.21.15.160:0/58208470 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3700050b0 con 0x7fd390105650 2024-09-16T08:40:00.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.593+0000 7fd37f7fe640 1 -- 172.21.15.160:0/58208470 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (unknown 1092875540 0 1702466736) 0x7fd380005b10 con 0x7fd390105650 2024-09-16T08:40:00.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.650+0000 7fd394b78640 1 -- 172.21.15.160:0/58208470 --> v1:172.21.15.160:6789/0 -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fd3700053a0 con 0x7fd390105650 2024-09-16T08:40:00.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.652+0000 7fd37f7fe640 1 -- 172.21.15.160:0/58208470 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (unknown 534127980 0 0) 0x7fd380005650 con 0x7fd390105650 2024-09-16T08:40:00.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.653+0000 7fd394b78640 1 -- 172.21.15.160:0/58208470 >> v1:172.21.15.160:6789/0 conn(0x7fd390105650 legacy=0x7fd3900797c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:00.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.653+0000 7fd394b78640 1 -- 172.21.15.160:0/58208470 shutdown_connections 2024-09-16T08:40:00.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.653+0000 7fd394b78640 1 -- 172.21.15.160:0/58208470 >> 172.21.15.160:0/58208470 conn(0x7fd390100f80 msgr2=0x7fd3901033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:00.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.653+0000 7fd394b78640 1 -- 172.21.15.160:0/58208470 shutdown_connections 2024-09-16T08:40:00.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:39:59.653+0000 7fd394b78640 1 -- 172.21.15.160:0/58208470 wait complete. 2024-09-16T08:40:00.278 INFO:teuthology.orchestra.run.smithi160.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-16T08:40:00.309 INFO:teuthology.orchestra.run.smithi160.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-16T08:40:00.309 INFO:teuthology.orchestra.run.smithi160.stdout:Creating mgr... 2024-09-16T08:40:00.309 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-16T08:40:00.310 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-16T08:40:00.571 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mgr.y 2024-09-16T08:40:00.572 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Failed to reset failed state of unit ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mgr.y.service: Unit ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mgr.y.service not loaded. 2024-09-16T08:40:00.728 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-27cca2fc-7407-11ef-bceb-c7b262605968.target.wants/ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-27cca2fc-7407-11ef-bceb-c7b262605968@.service. 2024-09-16T08:40:01.273 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:01 smithi160 podman[30382]: 2024-09-16 08:40:01.20181221 +0000 UTC m=+0.343623755 container start 2dab0971fcc381fe821a07e377e243b621675202314f722d7bc826dc411b76ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-16T08:40:01.273 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:01 smithi160 bash[30382]: 2dab0971fcc381fe821a07e377e243b621675202314f722d7bc826dc411b76ee 2024-09-16T08:40:01.286 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T08:40:01.286 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout disabled 2024-09-16T08:40:01.293 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T08:40:01.294 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-09-16T08:40:01.294 INFO:teuthology.orchestra.run.smithi160.stdout:firewalld.service is not enabled 2024-09-16T08:40:01.294 INFO:teuthology.orchestra.run.smithi160.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T08:40:01.302 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T08:40:01.302 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout disabled 2024-09-16T08:40:01.310 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T08:40:01.310 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-09-16T08:40:01.311 INFO:teuthology.orchestra.run.smithi160.stdout:firewalld.service is not enabled 2024-09-16T08:40:01.311 INFO:teuthology.orchestra.run.smithi160.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-16T08:40:01.311 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mgr to start... 2024-09-16T08:40:01.311 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mgr... 2024-09-16T08:40:01.542 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:01 smithi160 systemd[1]: Started Ceph mgr.y for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:40:02.186 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3859442613' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-16T08:40:02.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-09-16T08:40:02.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:40:02.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsid": "27cca2fc-7407-11ef-bceb-c7b262605968", 2024-09-16T08:40:02.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T08:40:02.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T08:40:02.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T08:40:02.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T08:40:02.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:40:02.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T08:40:02.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T08:40:02.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 0 2024-09-16T08:40:02.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:40:02.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T08:40:02.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T08:40:02.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:40:02.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-16T08:40:02.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T08:40:02.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:40:02.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T08:40:02.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T08:40:02.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:40:02.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T08:40:02.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:40:02.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T08:40:02.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T08:40:02.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T08:40:02.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T08:40:02.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T08:40:02.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T08:40:02.498 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:40:02.498 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T08:40:02.498 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T08:40:02.498 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T08:40:02.498 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T08:40:02.498 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T08:40:02.499 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T08:40:02.499 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T08:40:02.499 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T08:40:02.499 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T08:40:02.499 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:40:02.499 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T08:40:02.499 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:40:02.499 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T08:39:54:060588+0000", 2024-09-16T08:40:02.499 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T08:40:02.499 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T08:40:02.499 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:40:02.499 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T08:40:02.500 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T08:40:02.500 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T08:40:02.500 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T08:40:02.500 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T08:40:02.500 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T08:40:02.500 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T08:40:02.500 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:40:02.500 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T08:40:02.500 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:40:02.500 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T08:40:02.500 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:40:02.500 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T08:39:54.061240+0000", 2024-09-16T08:40:02.501 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T08:40:02.501 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:40:02.501 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T08:40:02.501 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:40:02.501 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.805+0000 7f9ba787b640 1 Processor -- start 2024-09-16T08:40:02.501 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.806+0000 7f9ba787b640 1 -- start start 2024-09-16T08:40:02.501 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.806+0000 7f9ba787b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9ba0079e90 con 0x7f9ba0079a00 2024-09-16T08:40:02.501 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.806+0000 7f9ba5617640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f9ba0079a00 0x7f9ba010b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:39832/0 (socket says 172.21.15.160:39832) 2024-09-16T08:40:02.501 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.806+0000 7f9ba5617640 1 -- 172.21.15.160:0/1259930529 learned_addr learned my addr 172.21.15.160:0/1259930529 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:02.501 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.806+0000 7f9b97fff640 1 -- 172.21.15.160:0/1259930529 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2792259013 0 0) 0x7f9b9c001110 con 0x7f9ba0079a00 2024-09-16T08:40:02.501 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.806+0000 7f9b97fff640 1 -- 172.21.15.160:0/1259930529 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b8c003520 con 0x7f9ba0079a00 2024-09-16T08:40:02.501 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.807+0000 7f9b97fff640 1 -- 172.21.15.160:0/1259930529 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1034335481 0 0) 0x7f9b8c003520 con 0x7f9ba0079a00 2024-09-16T08:40:02.502 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.807+0000 7f9b97fff640 1 -- 172.21.15.160:0/1259930529 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba010dee0 con 0x7f9ba0079a00 2024-09-16T08:40:02.502 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.807+0000 7f9b97fff640 1 -- 172.21.15.160:0/1259930529 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f9b9c0017d0 con 0x7f9ba0079a00 2024-09-16T08:40:02.502 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.807+0000 7f9b97fff640 1 -- 172.21.15.160:0/1259930529 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f9b9c0048a0 con 0x7f9ba0079a00 2024-09-16T08:40:02.502 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.807+0000 7f9b97fff640 1 -- 172.21.15.160:0/1259930529 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f9b9c0059e0 con 0x7f9ba0079a00 2024-09-16T08:40:02.502 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.808+0000 7f9ba787b640 1 -- 172.21.15.160:0/1259930529 >> v1:172.21.15.160:6789/0 conn(0x7f9ba0079a00 legacy=0x7f9ba010b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:02.502 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.808+0000 7f9ba787b640 1 -- 172.21.15.160:0/1259930529 shutdown_connections 2024-09-16T08:40:02.502 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.808+0000 7f9ba787b640 1 -- 172.21.15.160:0/1259930529 >> 172.21.15.160:0/1259930529 conn(0x7f9ba0100f80 msgr2=0x7f9ba01033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:02.502 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.808+0000 7f9ba787b640 1 -- 172.21.15.160:0/1259930529 shutdown_connections 2024-09-16T08:40:02.502 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.808+0000 7f9ba787b640 1 -- 172.21.15.160:0/1259930529 wait complete. 2024-09-16T08:40:02.503 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.808+0000 7f9ba787b640 1 Processor -- start 2024-09-16T08:40:02.503 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.809+0000 7f9ba787b640 1 -- start start 2024-09-16T08:40:02.503 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.809+0000 7f9ba787b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9ba019cf60 con 0x7f9ba0079a00 2024-09-16T08:40:02.503 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.809+0000 7f9ba5617640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f9ba0079a00 0x7f9ba019c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:39840/0 (socket says 172.21.15.160:39840) 2024-09-16T08:40:02.503 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.809+0000 7f9ba5617640 1 -- 172.21.15.160:0/3859442613 learned_addr learned my addr 172.21.15.160:0/3859442613 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:02.503 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.809+0000 7f9b967fc640 1 -- 172.21.15.160:0/3859442613 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 339424341 0 0) 0x7f9ba019cf60 con 0x7f9ba0079a00 2024-09-16T08:40:02.503 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.809+0000 7f9b967fc640 1 -- 172.21.15.160:0/3859442613 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b8c0055c0 con 0x7f9ba0079a00 2024-09-16T08:40:02.503 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.809+0000 7f9b967fc640 1 -- 172.21.15.160:0/3859442613 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2605398534 0 0) 0x7f9b8c0055c0 con 0x7f9ba0079a00 2024-09-16T08:40:02.503 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.809+0000 7f9b967fc640 1 -- 172.21.15.160:0/3859442613 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9ba019cf60 con 0x7f9ba0079a00 2024-09-16T08:40:02.503 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.809+0000 7f9b967fc640 1 -- 172.21.15.160:0/3859442613 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f9b9c001080 con 0x7f9ba0079a00 2024-09-16T08:40:02.503 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.809+0000 7f9b967fc640 1 -- 172.21.15.160:0/3859442613 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2865216874 0 0) 0x7f9ba019cf60 con 0x7f9ba0079a00 2024-09-16T08:40:02.503 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.809+0000 7f9b967fc640 1 -- 172.21.15.160:0/3859442613 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba019d150 con 0x7f9ba0079a00 2024-09-16T08:40:02.504 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.809+0000 7f9ba787b640 1 -- 172.21.15.160:0/3859442613 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba01a0a30 con 0x7f9ba0079a00 2024-09-16T08:40:02.504 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.810+0000 7f9ba787b640 1 -- 172.21.15.160:0/3859442613 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9ba01a0e50 con 0x7f9ba0079a00 2024-09-16T08:40:02.504 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.810+0000 7f9b967fc640 1 -- 172.21.15.160:0/3859442613 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f9b9c0048a0 con 0x7f9ba0079a00 2024-09-16T08:40:02.504 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.810+0000 7f9b967fc640 1 -- 172.21.15.160:0/3859442613 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f9b9c005e90 con 0x7f9ba0079a00 2024-09-16T08:40:02.504 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.810+0000 7f9b967fc640 1 -- 172.21.15.160:0/3859442613 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f9b9c006480 con 0x7f9ba0079a00 2024-09-16T08:40:02.504 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.810+0000 7f9b967fc640 1 -- 172.21.15.160:0/3859442613 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3090976605 0 0) 0x7f9b9c0079d0 con 0x7f9ba0079a00 2024-09-16T08:40:02.504 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.811+0000 7f9ba787b640 1 -- 172.21.15.160:0/3859442613 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b840050b0 con 0x7f9ba0079a00 2024-09-16T08:40:02.505 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.813+0000 7f9b967fc640 1 -- 172.21.15.160:0/3859442613 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (unknown 1092875540 0 1702466736) 0x7f9ba0079e90 con 0x7f9ba0079a00 2024-09-16T08:40:02.505 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.875+0000 7f9ba787b640 1 -- 172.21.15.160:0/3859442613 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9b840053a0 con 0x7f9ba0079a00 2024-09-16T08:40:02.506 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.875+0000 7f9b967fc640 1 -- 172.21.15.160:0/3859442613 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (unknown 4201413639 0 2438061538) 0x7f9b9c004e80 con 0x7f9ba0079a00 2024-09-16T08:40:02.506 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.876+0000 7f9ba787b640 1 -- 172.21.15.160:0/3859442613 >> v1:172.21.15.160:6789/0 conn(0x7f9ba0079a00 legacy=0x7f9ba019c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:02.506 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.876+0000 7f9ba787b640 1 -- 172.21.15.160:0/3859442613 shutdown_connections 2024-09-16T08:40:02.506 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.876+0000 7f9ba787b640 1 -- 172.21.15.160:0/3859442613 >> 172.21.15.160:0/3859442613 conn(0x7f9ba0100f80 msgr2=0x7f9ba0193200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:02.506 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.877+0000 7f9ba787b640 1 -- 172.21.15.160:0/3859442613 shutdown_connections 2024-09-16T08:40:02.506 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:01.877+0000 7f9ba787b640 1 -- 172.21.15.160:0/3859442613 wait complete. 2024-09-16T08:40:02.506 INFO:teuthology.orchestra.run.smithi160.stdout:mgr not available, waiting (1/15)... 2024-09-16T08:40:06.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:06 smithi160 ceph-mon[30137]: Activating manager daemon y 2024-09-16T08:40:06.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:06 smithi160 ceph-mon[30137]: mgrmap e2: y(active, starting, since 0.00319033s) 2024-09-16T08:40:06.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:06 smithi160 ceph-mon[30137]: from='mgr.14100 v1:172.21.15.160:0/1056911120' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-16T08:40:06.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:06 smithi160 ceph-mon[30137]: from='mgr.14100 v1:172.21.15.160:0/1056911120' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-16T08:40:06.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:06 smithi160 ceph-mon[30137]: from='mgr.14100 v1:172.21.15.160:0/1056911120' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-16T08:40:06.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:06 smithi160 ceph-mon[30137]: from='mgr.14100 v1:172.21.15.160:0/1056911120' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-16T08:40:06.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:06 smithi160 ceph-mon[30137]: from='mgr.14100 v1:172.21.15.160:0/1056911120' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-16T08:40:06.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:06 smithi160 ceph-mon[30137]: from='mgr.14100 v1:172.21.15.160:0/1056911120' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-16T08:40:06.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:06 smithi160 ceph-mon[30137]: from='mgr.14100 v1:172.21.15.160:0/1056911120' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:40:06.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:06 smithi160 ceph-mon[30137]: from='mgr.14100 v1:172.21.15.160:0/1056911120' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-16T08:40:06.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:06 smithi160 ceph-mon[30137]: Manager daemon y is now available 2024-09-16T08:40:06.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:06 smithi160 ceph-mon[30137]: from='mgr.14100 v1:172.21.15.160:0/1056911120' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-16T08:40:06.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:06 smithi160 ceph-mon[30137]: from='mgr.14100 v1:172.21.15.160:0/1056911120' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-16T08:40:06.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:06 smithi160 ceph-mon[30137]: from='mgr.14100 v1:172.21.15.160:0/1056911120' entity='mgr.y' 2024-09-16T08:40:06.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:06 smithi160 ceph-mon[30137]: from='mgr.14100 v1:172.21.15.160:0/1056911120' entity='mgr.y' 2024-09-16T08:40:06.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:06 smithi160 ceph-mon[30137]: from='mgr.14100 v1:172.21.15.160:0/1056911120' entity='mgr.y' 2024-09-16T08:40:08.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:08 smithi160 ceph-mon[30137]: mgrmap e3: y(active, since 1.00728s) 2024-09-16T08:40:09.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:09 smithi160 ceph-mon[30137]: mgrmap e4: y(active, since 2s) 2024-09-16T08:40:09.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1809326306' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-16T08:40:13.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-09-16T08:40:13.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:40:13.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsid": "27cca2fc-7407-11ef-bceb-c7b262605968", 2024-09-16T08:40:13.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T08:40:13.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T08:40:13.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T08:40:13.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T08:40:13.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:40:13.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T08:40:13.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T08:40:13.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 0 2024-09-16T08:40:13.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:40:13.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T08:40:13.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T08:40:13.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:40:13.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-16T08:40:13.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T08:40:13.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:40:13.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T08:40:13.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T08:40:13.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:40:13.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T08:40:13.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:40:13.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T08:40:13.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T08:40:13.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T08:40:13.915 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T08:40:13.915 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T08:40:13.916 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T08:40:13.916 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:40:13.916 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T08:40:13.916 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T08:40:13.916 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T08:40:13.916 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T08:40:13.916 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T08:40:13.916 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T08:40:13.916 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T08:40:13.916 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T08:40:13.916 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T08:40:13.916 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:40:13.916 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T08:40:13.917 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:40:13.917 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T08:39:54:060588+0000", 2024-09-16T08:40:13.917 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T08:40:13.917 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T08:40:13.917 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:40:13.917 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T08:40:13.917 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T08:40:13.917 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T08:40:13.917 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T08:40:13.917 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T08:40:13.917 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T08:40:13.917 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T08:40:13.917 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:40:13.918 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T08:40:13.918 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:40:13.918 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T08:40:13.918 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:40:13.918 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T08:39:54.061240+0000", 2024-09-16T08:40:13.918 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T08:40:13.918 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:40:13.918 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T08:40:13.918 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:40:13.918 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.271+0000 7f1809f35640 1 Processor -- start 2024-09-16T08:40:13.918 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.272+0000 7f1809f35640 1 -- start start 2024-09-16T08:40:13.918 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.272+0000 7f1809f35640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f180410aba0 con 0x7f1804106720 2024-09-16T08:40:13.919 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.272+0000 7f18037fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f1804106720 0x7f1804106b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:33746/0 (socket says 172.21.15.160:33746) 2024-09-16T08:40:13.919 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.272+0000 7f18037fe640 1 -- 172.21.15.160:0/3100239244 learned_addr learned my addr 172.21.15.160:0/3100239244 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:13.919 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.274+0000 7f18027fc640 1 -- 172.21.15.160:0/3100239244 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1888503990 0 0) 0x7f180410aba0 con 0x7f1804106720 2024-09-16T08:40:13.919 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.274+0000 7f18027fc640 1 -- 172.21.15.160:0/3100239244 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17ec003520 con 0x7f1804106720 2024-09-16T08:40:13.919 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.274+0000 7f18027fc640 1 -- 172.21.15.160:0/3100239244 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4214548932 0 0) 0x7f17ec003520 con 0x7f1804106720 2024-09-16T08:40:13.919 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.274+0000 7f18027fc640 1 -- 172.21.15.160:0/3100239244 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180410bd80 con 0x7f1804106720 2024-09-16T08:40:13.919 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.274+0000 7f18027fc640 1 -- 172.21.15.160:0/3100239244 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f17f4001600 con 0x7f1804106720 2024-09-16T08:40:13.919 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.274+0000 7f18027fc640 1 -- 172.21.15.160:0/3100239244 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f17f4001cf0 con 0x7f1804106720 2024-09-16T08:40:13.919 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.274+0000 7f18027fc640 1 -- 172.21.15.160:0/3100239244 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f17f40058d0 con 0x7f1804106720 2024-09-16T08:40:13.919 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.275+0000 7f1809f35640 1 -- 172.21.15.160:0/3100239244 >> v1:172.21.15.160:6789/0 conn(0x7f1804106720 legacy=0x7f1804106b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:13.919 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.275+0000 7f1809f35640 1 -- 172.21.15.160:0/3100239244 shutdown_connections 2024-09-16T08:40:13.919 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.275+0000 7f1809f35640 1 -- 172.21.15.160:0/3100239244 >> 172.21.15.160:0/3100239244 conn(0x7f1804102070 msgr2=0x7f1804104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:13.920 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.275+0000 7f1809f35640 1 -- 172.21.15.160:0/3100239244 shutdown_connections 2024-09-16T08:40:13.920 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.275+0000 7f1809f35640 1 -- 172.21.15.160:0/3100239244 wait complete. 2024-09-16T08:40:13.920 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.276+0000 7f1809f35640 1 Processor -- start 2024-09-16T08:40:13.920 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.276+0000 7f1809f35640 1 -- start start 2024-09-16T08:40:13.920 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.276+0000 7f1809f35640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f180419aa20 con 0x7f1804106720 2024-09-16T08:40:13.920 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.276+0000 7f18037fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f1804106720 0x7f180419a2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:33748/0 (socket says 172.21.15.160:33748) 2024-09-16T08:40:13.920 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.276+0000 7f18037fe640 1 -- 172.21.15.160:0/1809326306 learned_addr learned my addr 172.21.15.160:0/1809326306 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:13.920 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.276+0000 7f1800ff9640 1 -- 172.21.15.160:0/1809326306 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1182497742 0 0) 0x7f180419aa20 con 0x7f1804106720 2024-09-16T08:40:13.920 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.276+0000 7f1800ff9640 1 -- 172.21.15.160:0/1809326306 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17ec005570 con 0x7f1804106720 2024-09-16T08:40:13.920 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.277+0000 7f1800ff9640 1 -- 172.21.15.160:0/1809326306 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 497508204 0 0) 0x7f17ec005570 con 0x7f1804106720 2024-09-16T08:40:13.920 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.277+0000 7f1800ff9640 1 -- 172.21.15.160:0/1809326306 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f180419aa20 con 0x7f1804106720 2024-09-16T08:40:13.920 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.277+0000 7f1800ff9640 1 -- 172.21.15.160:0/1809326306 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f17f4001080 con 0x7f1804106720 2024-09-16T08:40:13.921 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.277+0000 7f1800ff9640 1 -- 172.21.15.160:0/1809326306 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4279207762 0 0) 0x7f180419aa20 con 0x7f1804106720 2024-09-16T08:40:13.921 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.277+0000 7f1800ff9640 1 -- 172.21.15.160:0/1809326306 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180419ac10 con 0x7f1804106720 2024-09-16T08:40:13.921 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.277+0000 7f1809f35640 1 -- 172.21.15.160:0/1809326306 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f180419e4f0 con 0x7f1804106720 2024-09-16T08:40:13.921 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.277+0000 7f1809f35640 1 -- 172.21.15.160:0/1809326306 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f180419e910 con 0x7f1804106720 2024-09-16T08:40:13.921 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.277+0000 7f1800ff9640 1 -- 172.21.15.160:0/1809326306 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f17f4001c30 con 0x7f1804106720 2024-09-16T08:40:13.921 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.277+0000 7f1800ff9640 1 -- 172.21.15.160:0/1809326306 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f17f4005fb0 con 0x7f1804106720 2024-09-16T08:40:13.921 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.278+0000 7f1800ff9640 1 -- 172.21.15.160:0/1809326306 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 4) ==== 51719+0+0 (unknown 1102428153 0 0) 0x7f17f4012be0 con 0x7f1804106720 2024-09-16T08:40:13.922 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.278+0000 7f1800ff9640 1 -- 172.21.15.160:0/1809326306 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3090976605 0 0) 0x7f17f4050420 con 0x7f1804106720 2024-09-16T08:40:13.923 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.278+0000 7f1809f35640 1 -- 172.21.15.160:0/1809326306 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f00050b0 con 0x7f1804106720 2024-09-16T08:40:13.923 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.283+0000 7f1800ff9640 1 -- 172.21.15.160:0/1809326306 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f17f4018df0 con 0x7f1804106720 2024-09-16T08:40:13.923 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.526+0000 7f1809f35640 1 -- 172.21.15.160:0/1809326306 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f17f00053a0 con 0x7f1804106720 2024-09-16T08:40:13.923 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.527+0000 7f1800ff9640 1 -- 172.21.15.160:0/1809326306 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (unknown 4201413639 0 2114801533) 0x7f17f40186f0 con 0x7f1804106720 2024-09-16T08:40:13.923 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.529+0000 7f1809f35640 1 -- 172.21.15.160:0/1809326306 >> v1:172.21.15.160:6800/2809527782 conn(0x7f17ec040800 legacy=0x7f17ec042cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:13.923 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.529+0000 7f1809f35640 1 -- 172.21.15.160:0/1809326306 >> v1:172.21.15.160:6789/0 conn(0x7f1804106720 legacy=0x7f180419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:13.923 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.529+0000 7f1809f35640 1 -- 172.21.15.160:0/1809326306 shutdown_connections 2024-09-16T08:40:13.923 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.529+0000 7f1809f35640 1 -- 172.21.15.160:0/1809326306 >> 172.21.15.160:0/1809326306 conn(0x7f1804102070 msgr2=0x7f1804104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:13.923 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.529+0000 7f1809f35640 1 -- 172.21.15.160:0/1809326306 shutdown_connections 2024-09-16T08:40:13.923 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:08.529+0000 7f1809f35640 1 -- 172.21.15.160:0/1809326306 wait complete. 2024-09-16T08:40:13.923 INFO:teuthology.orchestra.run.smithi160.stdout:mgr is available 2024-09-16T08:40:14.831 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/941543417' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-16T08:40:15.124 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-09-16T08:40:15.125 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T08:40:15.125 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout fsid = 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:40:15.125 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-16T08:40:15.125 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_host = [v1:172.21.15.160:6789] 2024-09-16T08:40:15.125 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T08:40:15.125 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T08:40:15.125 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T08:40:15.125 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T08:40:15.125 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-09-16T08:40:15.126 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T08:40:15.126 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T08:40:15.126 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-09-16T08:40:15.126 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T08:40:15.126 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T08:40:15.126 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T08:40:15.126 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.345+0000 7f2e27566640 1 Processor -- start 2024-09-16T08:40:15.126 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.345+0000 7f2e27566640 1 -- start start 2024-09-16T08:40:15.126 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.345+0000 7f2e27566640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2e2010aba0 con 0x7f2e20106720 2024-09-16T08:40:15.126 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.346+0000 7f2e25302640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f2e20106720 0x7f2e20106b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:33756/0 (socket says 172.21.15.160:33756) 2024-09-16T08:40:15.126 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.346+0000 7f2e25302640 1 -- 172.21.15.160:0/4018417361 learned_addr learned my addr 172.21.15.160:0/4018417361 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:15.126 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.346+0000 7f2e17fff640 1 -- 172.21.15.160:0/4018417361 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2435915252 0 0) 0x7f2e2010aba0 con 0x7f2e20106720 2024-09-16T08:40:15.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.346+0000 7f2e17fff640 1 -- 172.21.15.160:0/4018417361 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e0c003520 con 0x7f2e20106720 2024-09-16T08:40:15.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.346+0000 7f2e17fff640 1 -- 172.21.15.160:0/4018417361 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3294552843 0 0) 0x7f2e0c003520 con 0x7f2e20106720 2024-09-16T08:40:15.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.346+0000 7f2e17fff640 1 -- 172.21.15.160:0/4018417361 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2010bd80 con 0x7f2e20106720 2024-09-16T08:40:15.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.346+0000 7f2e17fff640 1 -- 172.21.15.160:0/4018417361 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f2e1c001980 con 0x7f2e20106720 2024-09-16T08:40:15.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.346+0000 7f2e17fff640 1 -- 172.21.15.160:0/4018417361 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f2e1c002070 con 0x7f2e20106720 2024-09-16T08:40:15.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.346+0000 7f2e17fff640 1 -- 172.21.15.160:0/4018417361 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f2e1c005c50 con 0x7f2e20106720 2024-09-16T08:40:15.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.347+0000 7f2e27566640 1 -- 172.21.15.160:0/4018417361 >> v1:172.21.15.160:6789/0 conn(0x7f2e20106720 legacy=0x7f2e20106b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:15.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.347+0000 7f2e27566640 1 -- 172.21.15.160:0/4018417361 shutdown_connections 2024-09-16T08:40:15.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.347+0000 7f2e27566640 1 -- 172.21.15.160:0/4018417361 >> 172.21.15.160:0/4018417361 conn(0x7f2e20102070 msgr2=0x7f2e20104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:15.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.347+0000 7f2e27566640 1 -- 172.21.15.160:0/4018417361 shutdown_connections 2024-09-16T08:40:15.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.347+0000 7f2e27566640 1 -- 172.21.15.160:0/4018417361 wait complete. 2024-09-16T08:40:15.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.348+0000 7f2e27566640 1 Processor -- start 2024-09-16T08:40:15.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.348+0000 7f2e27566640 1 -- start start 2024-09-16T08:40:15.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.348+0000 7f2e27566640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2e2019aa20 con 0x7f2e20106720 2024-09-16T08:40:15.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.348+0000 7f2e25302640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f2e20106720 0x7f2e2019a2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:33766/0 (socket says 172.21.15.160:33766) 2024-09-16T08:40:15.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.348+0000 7f2e25302640 1 -- 172.21.15.160:0/941543417 learned_addr learned my addr 172.21.15.160:0/941543417 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:15.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.348+0000 7f2e167fc640 1 -- 172.21.15.160:0/941543417 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1710378342 0 0) 0x7f2e2019aa20 con 0x7f2e20106720 2024-09-16T08:40:15.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.348+0000 7f2e167fc640 1 -- 172.21.15.160:0/941543417 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e0c005570 con 0x7f2e20106720 2024-09-16T08:40:15.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.348+0000 7f2e167fc640 1 -- 172.21.15.160:0/941543417 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3050776837 0 0) 0x7f2e0c005570 con 0x7f2e20106720 2024-09-16T08:40:15.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.348+0000 7f2e167fc640 1 -- 172.21.15.160:0/941543417 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2e2019aa20 con 0x7f2e20106720 2024-09-16T08:40:15.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.348+0000 7f2e167fc640 1 -- 172.21.15.160:0/941543417 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f2e1c004b60 con 0x7f2e20106720 2024-09-16T08:40:15.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.348+0000 7f2e167fc640 1 -- 172.21.15.160:0/941543417 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 392357593 0 0) 0x7f2e2019aa20 con 0x7f2e20106720 2024-09-16T08:40:15.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.348+0000 7f2e167fc640 1 -- 172.21.15.160:0/941543417 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2019ac10 con 0x7f2e20106720 2024-09-16T08:40:15.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.348+0000 7f2e27566640 1 -- 172.21.15.160:0/941543417 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2e2019e4f0 con 0x7f2e20106720 2024-09-16T08:40:15.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.348+0000 7f2e27566640 1 -- 172.21.15.160:0/941543417 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f2e2019e910 con 0x7f2e20106720 2024-09-16T08:40:15.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.349+0000 7f2e167fc640 1 -- 172.21.15.160:0/941543417 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f2e1c001f30 con 0x7f2e20106720 2024-09-16T08:40:15.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.349+0000 7f2e167fc640 1 -- 172.21.15.160:0/941543417 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f2e1c006300 con 0x7f2e20106720 2024-09-16T08:40:15.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.350+0000 7f2e167fc640 1 -- 172.21.15.160:0/941543417 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 4) ==== 51719+0+0 (unknown 1102428153 0 0) 0x7f2e1c012eb0 con 0x7f2e20106720 2024-09-16T08:40:15.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.350+0000 7f2e27566640 1 -- 172.21.15.160:0/941543417 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e040050b0 con 0x7f2e20106720 2024-09-16T08:40:15.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.350+0000 7f2e167fc640 1 -- 172.21.15.160:0/941543417 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3090976605 0 0) 0x7f2e1c04f950 con 0x7f2e20106720 2024-09-16T08:40:15.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.354+0000 7f2e167fc640 1 -- 172.21.15.160:0/941543417 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f2e1c019070 con 0x7f2e20106720 2024-09-16T08:40:15.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.527+0000 7f2e27566640 1 -- 172.21.15.160:0/941543417 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f2e04002420 con 0x7f2e20106720 2024-09-16T08:40:15.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.528+0000 7f2e167fc640 1 -- 172.21.15.160:0/941543417 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+354 (unknown 1187553405 0 4075966108) 0x7f2e1c018970 con 0x7f2e20106720 2024-09-16T08:40:15.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.532+0000 7f2e27566640 1 -- 172.21.15.160:0/941543417 >> v1:172.21.15.160:6800/2809527782 conn(0x7f2e0c040870 legacy=0x7f2e0c042d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:15.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.532+0000 7f2e27566640 1 -- 172.21.15.160:0/941543417 >> v1:172.21.15.160:6789/0 conn(0x7f2e20106720 legacy=0x7f2e2019a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:15.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.532+0000 7f2e27566640 1 -- 172.21.15.160:0/941543417 shutdown_connections 2024-09-16T08:40:15.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.532+0000 7f2e27566640 1 -- 172.21.15.160:0/941543417 >> 172.21.15.160:0/941543417 conn(0x7f2e20102070 msgr2=0x7f2e20104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:15.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.532+0000 7f2e27566640 1 -- 172.21.15.160:0/941543417 shutdown_connections 2024-09-16T08:40:15.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:14.532+0000 7f2e27566640 1 -- 172.21.15.160:0/941543417 wait complete. 2024-09-16T08:40:15.130 INFO:teuthology.orchestra.run.smithi160.stdout:Enabling cephadm module... 2024-09-16T08:40:16.145 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:15 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2857672138' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-16T08:40:16.145 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:15 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ignoring --setuser ceph since I am not root 2024-09-16T08:40:16.146 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:15 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ignoring --setgroup ceph since I am not root 2024-09-16T08:40:16.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.555+0000 7f93797ad640 1 Processor -- start 2024-09-16T08:40:16.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.555+0000 7f93797ad640 1 -- start start 2024-09-16T08:40:16.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.555+0000 7f93797ad640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f937407b250 con 0x7f937407c4e0 2024-09-16T08:40:16.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.555+0000 7f9372ffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f937407c4e0 0x7f937407ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:33768/0 (socket says 172.21.15.160:33768) 2024-09-16T08:40:16.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.555+0000 7f9372ffd640 1 -- 172.21.15.160:0/2767397089 learned_addr learned my addr 172.21.15.160:0/2767397089 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:16.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.555+0000 7f9371ffb640 1 -- 172.21.15.160:0/2767397089 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3985197321 0 0) 0x7f937407b250 con 0x7f937407c4e0 2024-09-16T08:40:16.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.555+0000 7f9371ffb640 1 -- 172.21.15.160:0/2767397089 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f935c003520 con 0x7f937407c4e0 2024-09-16T08:40:16.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.556+0000 7f9371ffb640 1 -- 172.21.15.160:0/2767397089 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1009194406 0 0) 0x7f935c003520 con 0x7f937407c4e0 2024-09-16T08:40:16.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.556+0000 7f9371ffb640 1 -- 172.21.15.160:0/2767397089 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937407b420 con 0x7f937407c4e0 2024-09-16T08:40:16.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.556+0000 7f9371ffb640 1 -- 172.21.15.160:0/2767397089 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f9364001600 con 0x7f937407c4e0 2024-09-16T08:40:16.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.556+0000 7f9371ffb640 1 -- 172.21.15.160:0/2767397089 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f9364001cf0 con 0x7f937407c4e0 2024-09-16T08:40:16.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.556+0000 7f9371ffb640 1 -- 172.21.15.160:0/2767397089 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f93640058d0 con 0x7f937407c4e0 2024-09-16T08:40:16.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.556+0000 7f93797ad640 1 -- 172.21.15.160:0/2767397089 >> v1:172.21.15.160:6789/0 conn(0x7f937407c4e0 legacy=0x7f937407ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:16.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.556+0000 7f93797ad640 1 -- 172.21.15.160:0/2767397089 shutdown_connections 2024-09-16T08:40:16.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.556+0000 7f93797ad640 1 -- 172.21.15.160:0/2767397089 >> 172.21.15.160:0/2767397089 conn(0x7f9374102070 msgr2=0x7f9374104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:16.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.556+0000 7f93797ad640 1 -- 172.21.15.160:0/2767397089 shutdown_connections 2024-09-16T08:40:16.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.556+0000 7f93797ad640 1 -- 172.21.15.160:0/2767397089 wait complete. 2024-09-16T08:40:16.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.557+0000 7f93797ad640 1 Processor -- start 2024-09-16T08:40:16.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.557+0000 7f93797ad640 1 -- start start 2024-09-16T08:40:16.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.557+0000 7f93797ad640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f937419cc20 con 0x7f937407c4e0 2024-09-16T08:40:16.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.557+0000 7f9372ffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f937407c4e0 0x7f937419c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:33780/0 (socket says 172.21.15.160:33780) 2024-09-16T08:40:16.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.557+0000 7f9372ffd640 1 -- 172.21.15.160:0/2857672138 learned_addr learned my addr 172.21.15.160:0/2857672138 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:16.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.558+0000 7f935bfff640 1 -- 172.21.15.160:0/2857672138 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3853599310 0 0) 0x7f937419cc20 con 0x7f937407c4e0 2024-09-16T08:40:16.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.558+0000 7f935bfff640 1 -- 172.21.15.160:0/2857672138 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f935c005570 con 0x7f937407c4e0 2024-09-16T08:40:16.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.558+0000 7f935bfff640 1 -- 172.21.15.160:0/2857672138 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 212907245 0 0) 0x7f935c005570 con 0x7f937407c4e0 2024-09-16T08:40:16.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.558+0000 7f935bfff640 1 -- 172.21.15.160:0/2857672138 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f937419cc20 con 0x7f937407c4e0 2024-09-16T08:40:16.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.558+0000 7f935bfff640 1 -- 172.21.15.160:0/2857672138 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f9364001080 con 0x7f937407c4e0 2024-09-16T08:40:16.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.558+0000 7f935bfff640 1 -- 172.21.15.160:0/2857672138 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4015986767 0 0) 0x7f937419cc20 con 0x7f937407c4e0 2024-09-16T08:40:16.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.558+0000 7f935bfff640 1 -- 172.21.15.160:0/2857672138 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937419ce10 con 0x7f937407c4e0 2024-09-16T08:40:16.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.558+0000 7f93797ad640 1 -- 172.21.15.160:0/2857672138 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f93741a06f0 con 0x7f937407c4e0 2024-09-16T08:40:16.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.558+0000 7f93797ad640 1 -- 172.21.15.160:0/2857672138 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f93741a0b10 con 0x7f937407c4e0 2024-09-16T08:40:16.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.558+0000 7f935bfff640 1 -- 172.21.15.160:0/2857672138 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f9364001c30 con 0x7f937407c4e0 2024-09-16T08:40:16.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.558+0000 7f935bfff640 1 -- 172.21.15.160:0/2857672138 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f9364005fb0 con 0x7f937407c4e0 2024-09-16T08:40:16.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.559+0000 7f935bfff640 1 -- 172.21.15.160:0/2857672138 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 4) ==== 51719+0+0 (unknown 1102428153 0 0) 0x7f9364012be0 con 0x7f937407c4e0 2024-09-16T08:40:16.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.559+0000 7f93797ad640 1 -- 172.21.15.160:0/2857672138 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93600050b0 con 0x7f937407c4e0 2024-09-16T08:40:16.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.559+0000 7f935bfff640 1 -- 172.21.15.160:0/2857672138 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3090976605 0 0) 0x7f936404f610 con 0x7f937407c4e0 2024-09-16T08:40:16.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.564+0000 7f935bfff640 1 -- 172.21.15.160:0/2857672138 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f9364018df0 con 0x7f937407c4e0 2024-09-16T08:40:16.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.784+0000 7f93797ad640 1 -- 172.21.15.160:0/2857672138 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f93600053a0 con 0x7f937407c4e0 2024-09-16T08:40:16.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.835+0000 7f935bfff640 1 -- 172.21.15.160:0/2857672138 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (unknown 2263024820 0 0) 0x7f93640186f0 con 0x7f937407c4e0 2024-09-16T08:40:16.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.837+0000 7f935bfff640 1 -- 172.21.15.160:0/2857672138 <== mon.0 v1:172.21.15.160:6789/0 11 ==== mgrmap(e 5) ==== 51730+0+0 (unknown 390724134 0 0) 0x7f936404e000 con 0x7f937407c4e0 2024-09-16T08:40:16.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.838+0000 7f93797ad640 1 -- 172.21.15.160:0/2857672138 >> v1:172.21.15.160:6800/2809527782 conn(0x7f935c040800 legacy=0x7f935c042cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:16.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.838+0000 7f93797ad640 1 -- 172.21.15.160:0/2857672138 >> v1:172.21.15.160:6789/0 conn(0x7f937407c4e0 legacy=0x7f937419c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:16.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.838+0000 7f93797ad640 1 -- 172.21.15.160:0/2857672138 shutdown_connections 2024-09-16T08:40:16.498 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.838+0000 7f93797ad640 1 -- 172.21.15.160:0/2857672138 >> 172.21.15.160:0/2857672138 conn(0x7f9374102070 msgr2=0x7f937410ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:16.498 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.838+0000 7f93797ad640 1 -- 172.21.15.160:0/2857672138 shutdown_connections 2024-09-16T08:40:16.498 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:15.838+0000 7f93797ad640 1 -- 172.21.15.160:0/2857672138 wait complete. 2024-09-16T08:40:17.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2857672138' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-16T08:40:17.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:16 smithi160 ceph-mon[30137]: mgrmap e5: y(active, since 9s) 2024-09-16T08:40:18.160 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/617591719' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-16T08:40:18.386 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:40:18.386 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-16T08:40:18.386 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T08:40:18.386 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-16T08:40:18.387 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T08:40:18.387 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:40:18.387 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.620+0000 7fdf47c94640 1 Processor -- start 2024-09-16T08:40:18.387 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.620+0000 7fdf47c94640 1 -- start start 2024-09-16T08:40:18.387 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.620+0000 7fdf47c94640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fdf4010aae0 con 0x7fdf40079b90 2024-09-16T08:40:18.387 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.620+0000 7fdf45a30640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fdf40079b90 0x7fdf40109330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36328/0 (socket says 172.21.15.160:36328) 2024-09-16T08:40:18.387 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.620+0000 7fdf45a30640 1 -- 172.21.15.160:0/268990981 learned_addr learned my addr 172.21.15.160:0/268990981 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:18.387 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.621+0000 7fdf44a2e640 1 -- 172.21.15.160:0/268990981 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1537964964 0 0) 0x7fdf4010aae0 con 0x7fdf40079b90 2024-09-16T08:40:18.387 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.621+0000 7fdf44a2e640 1 -- 172.21.15.160:0/268990981 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf2c003520 con 0x7fdf40079b90 2024-09-16T08:40:18.387 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.621+0000 7fdf44a2e640 1 -- 172.21.15.160:0/268990981 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2079118415 0 0) 0x7fdf2c003520 con 0x7fdf40079b90 2024-09-16T08:40:18.387 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.621+0000 7fdf44a2e640 1 -- 172.21.15.160:0/268990981 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4010bcc0 con 0x7fdf40079b90 2024-09-16T08:40:18.387 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.621+0000 7fdf44a2e640 1 -- 172.21.15.160:0/268990981 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fdf3c001600 con 0x7fdf40079b90 2024-09-16T08:40:18.388 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.621+0000 7fdf44a2e640 1 -- 172.21.15.160:0/268990981 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7fdf3c001bf0 con 0x7fdf40079b90 2024-09-16T08:40:18.388 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.621+0000 7fdf44a2e640 1 -- 172.21.15.160:0/268990981 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fdf3c0058d0 con 0x7fdf40079b90 2024-09-16T08:40:18.388 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.622+0000 7fdf47c94640 1 -- 172.21.15.160:0/268990981 >> v1:172.21.15.160:6789/0 conn(0x7fdf40079b90 legacy=0x7fdf40109330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:18.388 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.622+0000 7fdf47c94640 1 -- 172.21.15.160:0/268990981 shutdown_connections 2024-09-16T08:40:18.388 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.622+0000 7fdf47c94640 1 -- 172.21.15.160:0/268990981 >> 172.21.15.160:0/268990981 conn(0x7fdf40100f80 msgr2=0x7fdf401033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:18.388 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.622+0000 7fdf47c94640 1 -- 172.21.15.160:0/268990981 shutdown_connections 2024-09-16T08:40:18.388 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.622+0000 7fdf47c94640 1 -- 172.21.15.160:0/268990981 wait complete. 2024-09-16T08:40:18.388 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.622+0000 7fdf47c94640 1 Processor -- start 2024-09-16T08:40:18.388 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.622+0000 7fdf47c94640 1 -- start start 2024-09-16T08:40:18.388 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.623+0000 7fdf47c94640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fdf4019ab50 con 0x7fdf40079b90 2024-09-16T08:40:18.388 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.623+0000 7fdf45a30640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fdf40079b90 0x7fdf4019a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36340/0 (socket says 172.21.15.160:36340) 2024-09-16T08:40:18.388 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.623+0000 7fdf45a30640 1 -- 172.21.15.160:0/617591719 learned_addr learned my addr 172.21.15.160:0/617591719 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:18.388 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.623+0000 7fdf36ffd640 1 -- 172.21.15.160:0/617591719 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1657853627 0 0) 0x7fdf4019ab50 con 0x7fdf40079b90 2024-09-16T08:40:18.388 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.623+0000 7fdf36ffd640 1 -- 172.21.15.160:0/617591719 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf2c0055c0 con 0x7fdf40079b90 2024-09-16T08:40:18.389 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.623+0000 7fdf36ffd640 1 -- 172.21.15.160:0/617591719 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 139206394 0 0) 0x7fdf2c0055c0 con 0x7fdf40079b90 2024-09-16T08:40:18.389 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.623+0000 7fdf36ffd640 1 -- 172.21.15.160:0/617591719 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdf4019ab50 con 0x7fdf40079b90 2024-09-16T08:40:18.389 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.623+0000 7fdf36ffd640 1 -- 172.21.15.160:0/617591719 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fdf3c001080 con 0x7fdf40079b90 2024-09-16T08:40:18.389 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.623+0000 7fdf36ffd640 1 -- 172.21.15.160:0/617591719 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3061828898 0 0) 0x7fdf4019ab50 con 0x7fdf40079b90 2024-09-16T08:40:18.389 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.623+0000 7fdf36ffd640 1 -- 172.21.15.160:0/617591719 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4019ad40 con 0x7fdf40079b90 2024-09-16T08:40:18.389 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.623+0000 7fdf47c94640 1 -- 172.21.15.160:0/617591719 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4019e620 con 0x7fdf40079b90 2024-09-16T08:40:18.389 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.623+0000 7fdf47c94640 1 -- 172.21.15.160:0/617591719 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fdf4019ea40 con 0x7fdf40079b90 2024-09-16T08:40:18.389 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.623+0000 7fdf36ffd640 1 -- 172.21.15.160:0/617591719 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7fdf3c0016c0 con 0x7fdf40079b90 2024-09-16T08:40:18.389 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.623+0000 7fdf36ffd640 1 -- 172.21.15.160:0/617591719 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fdf3c005ef0 con 0x7fdf40079b90 2024-09-16T08:40:18.389 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.624+0000 7fdf47c94640 1 -- 172.21.15.160:0/617591719 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf240050b0 con 0x7fdf40079b90 2024-09-16T08:40:18.389 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.624+0000 7fdf36ffd640 1 -- 172.21.15.160:0/617591719 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 5) ==== 51730+0+0 (unknown 390724134 0 0) 0x7fdf3c012b50 con 0x7fdf40079b90 2024-09-16T08:40:18.389 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.625+0000 7fdf36ffd640 1 -- 172.21.15.160:0/617591719 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3090976605 0 0) 0x7fdf3c04f5e0 con 0x7fdf40079b90 2024-09-16T08:40:18.390 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.625+0000 7fdf4522f640 1 -- 172.21.15.160:0/617591719 >> v1:172.21.15.160:6800/2809527782 conn(0x7fdf2c040830 legacy=0x7fdf2c042cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/2809527782 2024-09-16T08:40:18.390 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.629+0000 7fdf36ffd640 1 -- 172.21.15.160:0/617591719 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fdf3c018dd0 con 0x7fdf40079b90 2024-09-16T08:40:18.390 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.826+0000 7fdf4522f640 1 -- 172.21.15.160:0/617591719 >> v1:172.21.15.160:6800/2809527782 conn(0x7fdf2c040830 legacy=0x7fdf2c042cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/2809527782 2024-09-16T08:40:18.390 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.859+0000 7fdf47c94640 1 -- 172.21.15.160:0/617591719 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fdf24005c70 con 0x7fdf40079b90 2024-09-16T08:40:18.390 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.859+0000 7fdf36ffd640 1 -- 172.21.15.160:0/617591719 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (unknown 3768197548 0 15966916) 0x7fdf3c0186d0 con 0x7fdf40079b90 2024-09-16T08:40:18.390 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.862+0000 7fdf47c94640 1 -- 172.21.15.160:0/617591719 >> v1:172.21.15.160:6800/2809527782 conn(0x7fdf2c040830 legacy=0x7fdf2c042cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T08:40:18.390 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.862+0000 7fdf47c94640 1 -- 172.21.15.160:0/617591719 >> v1:172.21.15.160:6789/0 conn(0x7fdf40079b90 legacy=0x7fdf4019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:18.390 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.862+0000 7fdf47c94640 1 -- 172.21.15.160:0/617591719 shutdown_connections 2024-09-16T08:40:18.390 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.862+0000 7fdf47c94640 1 -- 172.21.15.160:0/617591719 >> 172.21.15.160:0/617591719 conn(0x7fdf40100f80 msgr2=0x7fdf40108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:18.390 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.862+0000 7fdf47c94640 1 -- 172.21.15.160:0/617591719 shutdown_connections 2024-09-16T08:40:18.390 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:17.862+0000 7fdf47c94640 1 -- 172.21.15.160:0/617591719 wait complete. 2024-09-16T08:40:18.391 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for the mgr to restart... 2024-09-16T08:40:18.391 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mgr epoch 5... 2024-09-16T08:40:21.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:20 smithi160 ceph-mon[30137]: Active manager daemon y restarted 2024-09-16T08:40:21.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:20 smithi160 ceph-mon[30137]: Activating manager daemon y 2024-09-16T08:40:21.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:20 smithi160 ceph-mon[30137]: osdmap e2: 0 total, 0 up, 0 in 2024-09-16T08:40:21.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:20 smithi160 ceph-mon[30137]: mgrmap e6: y(active, starting, since 0.00327426s) 2024-09-16T08:40:21.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:20 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:40:21.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:20 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-16T08:40:21.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:20 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-16T08:40:21.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:20 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-16T08:40:21.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:20 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-16T08:40:21.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:20 smithi160 ceph-mon[30137]: Manager daemon y is now available 2024-09-16T08:40:22.483 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' 2024-09-16T08:40:22.484 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' 2024-09-16T08:40:22.484 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-mon[30137]: Found migration_current of "None". Setting to last migration. 2024-09-16T08:40:22.484 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' 2024-09-16T08:40:22.484 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' 2024-09-16T08:40:22.484 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:40:22.484 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:40:22.484 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-16T08:40:22.484 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-16T08:40:22.484 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-mon[30137]: from='client.14120 v1:172.21.15.160:0/683085996' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T08:40:22.485 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-mon[30137]: mgrmap e7: y(active, since 1.00552s) 2024-09-16T08:40:22.485 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-mon[30137]: from='client.14120 v1:172.21.15.160:0/683085996' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T08:40:22.485 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: 2024-09-16T08:40:22.401+0000 7f94991ab640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:08:40:22] ENGINE Error in HTTPServer.serve 2024-09-16T08:40:22.485 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: Traceback (most recent call last): 2024-09-16T08:40:22.485 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:40:22.485 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: self._connections.run(self.expiration_interval) 2024-09-16T08:40:22.485 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:40:22.486 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: self._run(expiration_interval) 2024-09-16T08:40:22.486 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:40:22.486 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:40:22.486 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:40:22.486 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:40:22.486 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:40:22.487 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: s = self.context.wrap_socket( 2024-09-16T08:40:22.487 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:40:22.487 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: return self.sslsocket_class._create( 2024-09-16T08:40:22.487 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:40:22.487 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: self.do_handshake() 2024-09-16T08:40:22.487 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:40:22.487 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: self._sslobj.do_handshake() 2024-09-16T08:40:22.487 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:40:22.487 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:22 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: 2024-09-16T08:40:22.493 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:40:22.493 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-16T08:40:22.493 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T08:40:22.493 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:40:22.493 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.879+0000 7f9bf3124640 1 Processor -- start 2024-09-16T08:40:22.493 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.879+0000 7f9bf3124640 1 -- start start 2024-09-16T08:40:22.493 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.879+0000 7f9bf3124640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9bec10aad0 con 0x7f9bec079b90 2024-09-16T08:40:22.493 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.879+0000 7f9bf0ec0640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f9bec079b90 0x7f9bec109320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36348/0 (socket says 172.21.15.160:36348) 2024-09-16T08:40:22.493 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.879+0000 7f9bf0ec0640 1 -- 172.21.15.160:0/1928088583 learned_addr learned my addr 172.21.15.160:0/1928088583 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:22.493 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.879+0000 7f9beb7fe640 1 -- 172.21.15.160:0/1928088583 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 580557303 0 0) 0x7f9bec10aad0 con 0x7f9bec079b90 2024-09-16T08:40:22.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.879+0000 7f9beb7fe640 1 -- 172.21.15.160:0/1928088583 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bdc003520 con 0x7f9bec079b90 2024-09-16T08:40:22.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.879+0000 7f9beb7fe640 1 -- 172.21.15.160:0/1928088583 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1404854070 0 0) 0x7f9bdc003520 con 0x7f9bec079b90 2024-09-16T08:40:22.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.880+0000 7f9beb7fe640 1 -- 172.21.15.160:0/1928088583 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bec10bcb0 con 0x7f9bec079b90 2024-09-16T08:40:22.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.880+0000 7f9beb7fe640 1 -- 172.21.15.160:0/1928088583 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f9be0001600 con 0x7f9bec079b90 2024-09-16T08:40:22.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.880+0000 7f9beb7fe640 1 -- 172.21.15.160:0/1928088583 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f9be0001cf0 con 0x7f9bec079b90 2024-09-16T08:40:22.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.880+0000 7f9beb7fe640 1 -- 172.21.15.160:0/1928088583 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f9be00058d0 con 0x7f9bec079b90 2024-09-16T08:40:22.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.880+0000 7f9bf3124640 1 -- 172.21.15.160:0/1928088583 >> v1:172.21.15.160:6789/0 conn(0x7f9bec079b90 legacy=0x7f9bec109320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:22.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.880+0000 7f9bf3124640 1 -- 172.21.15.160:0/1928088583 shutdown_connections 2024-09-16T08:40:22.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.880+0000 7f9bf3124640 1 -- 172.21.15.160:0/1928088583 >> 172.21.15.160:0/1928088583 conn(0x7f9bec100fb0 msgr2=0x7f9bec1033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:22.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.880+0000 7f9bf3124640 1 -- 172.21.15.160:0/1928088583 shutdown_connections 2024-09-16T08:40:22.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.880+0000 7f9bf3124640 1 -- 172.21.15.160:0/1928088583 wait complete. 2024-09-16T08:40:22.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.881+0000 7f9bf3124640 1 Processor -- start 2024-09-16T08:40:22.494 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.881+0000 7f9bf3124640 1 -- start start 2024-09-16T08:40:22.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.881+0000 7f9bf3124640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9bec19aa30 con 0x7f9bec079b90 2024-09-16T08:40:22.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.882+0000 7f9bf0ec0640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f9bec079b90 0x7f9bec19a2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36354/0 (socket says 172.21.15.160:36354) 2024-09-16T08:40:22.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.882+0000 7f9bf0ec0640 1 -- 172.21.15.160:0/683085996 learned_addr learned my addr 172.21.15.160:0/683085996 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:22.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.882+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1370552175 0 0) 0x7f9bec19aa30 con 0x7f9bec079b90 2024-09-16T08:40:22.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.882+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bdc005570 con 0x7f9bec079b90 2024-09-16T08:40:22.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.882+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2383575697 0 0) 0x7f9bdc005570 con 0x7f9bec079b90 2024-09-16T08:40:22.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.882+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9bec19aa30 con 0x7f9bec079b90 2024-09-16T08:40:22.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.882+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f9be00047e0 con 0x7f9bec079b90 2024-09-16T08:40:22.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.882+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1950107373 0 0) 0x7f9bec19aa30 con 0x7f9bec079b90 2024-09-16T08:40:22.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.882+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bec19ac20 con 0x7f9bec079b90 2024-09-16T08:40:22.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.882+0000 7f9bf3124640 1 -- 172.21.15.160:0/683085996 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9bec19e500 con 0x7f9bec079b90 2024-09-16T08:40:22.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.882+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f9be0001b30 con 0x7f9bec079b90 2024-09-16T08:40:22.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.882+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f9be00061a0 con 0x7f9bec079b90 2024-09-16T08:40:22.495 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.882+0000 7f9bf3124640 1 -- 172.21.15.160:0/683085996 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9bec19e920 con 0x7f9bec079b90 2024-09-16T08:40:22.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.883+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 5) ==== 51730+0+0 (unknown 390724134 0 0) 0x7f9be0012e00 con 0x7f9bec079b90 2024-09-16T08:40:22.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.884+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 --> v1:172.21.15.160:6800/2809527782 -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9bdc044590 con 0x7f9bdc0408c0 2024-09-16T08:40:22.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.884+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3090976605 0 0) 0x7f9be004f590 con 0x7f9bec079b90 2024-09-16T08:40:22.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:18.884+0000 7f9bebfff640 1 -- 172.21.15.160:0/683085996 >> v1:172.21.15.160:6800/2809527782 conn(0x7f9bdc0408c0 legacy=0x7f9bdc042d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/2809527782 2024-09-16T08:40:22.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:19.085+0000 7f9bebfff640 1 -- 172.21.15.160:0/683085996 >> v1:172.21.15.160:6800/2809527782 conn(0x7f9bdc0408c0 legacy=0x7f9bdc042d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/2809527782 2024-09-16T08:40:22.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:19.487+0000 7f9bebfff640 1 -- 172.21.15.160:0/683085996 >> v1:172.21.15.160:6800/2809527782 conn(0x7f9bdc0408c0 legacy=0x7f9bdc042d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/2809527782 2024-09-16T08:40:22.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:20.288+0000 7f9bebfff640 1 -- 172.21.15.160:0/683085996 >> v1:172.21.15.160:6800/2809527782 conn(0x7f9bdc0408c0 legacy=0x7f9bdc042d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/2809527782 2024-09-16T08:40:22.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:20.862+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mgrmap(e 6) ==== 51532+0+0 (unknown 2365321671 0 0) 0x7f9be004df80 con 0x7f9bec079b90 2024-09-16T08:40:22.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:20.862+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 >> v1:172.21.15.160:6800/2809527782 conn(0x7f9bdc0408c0 legacy=0x7f9bdc042d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T08:40:22.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:21.863+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mgrmap(e 7) ==== 51624+0+0 (unknown 2115241582 0 0) 0x7f9be0050cf0 con 0x7f9bec079b90 2024-09-16T08:40:22.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:21.864+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 --> v1:172.21.15.160:6800/3425583652 -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9bdc044590 con 0x7f9bdc0450c0 2024-09-16T08:40:22.496 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:21.865+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 <== mgr.14116 v1:172.21.15.160:6800/3425583652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (unknown 0 0 1361822222) 0x7f9bdc044590 con 0x7f9bdc0450c0 2024-09-16T08:40:22.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:21.872+0000 7f9bf3124640 1 -- 172.21.15.160:0/683085996 --> v1:172.21.15.160:6800/3425583652 -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9bec10b870 con 0x7f9bdc0450c0 2024-09-16T08:40:22.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:21.872+0000 7f9be9ffb640 1 -- 172.21.15.160:0/683085996 <== mgr.14116 v1:172.21.15.160:6800/3425583652 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (unknown 0 0 96372106) 0x7f9bec10b870 con 0x7f9bdc0450c0 2024-09-16T08:40:22.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:21.872+0000 7f9bf3124640 1 -- 172.21.15.160:0/683085996 >> v1:172.21.15.160:6800/3425583652 conn(0x7f9bdc0450c0 legacy=0x7f9bdc0474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:22.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:21.872+0000 7f9bf3124640 1 -- 172.21.15.160:0/683085996 >> v1:172.21.15.160:6789/0 conn(0x7f9bec079b90 legacy=0x7f9bec19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:22.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:21.872+0000 7f9bf3124640 1 -- 172.21.15.160:0/683085996 shutdown_connections 2024-09-16T08:40:22.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:21.872+0000 7f9bf3124640 1 -- 172.21.15.160:0/683085996 >> 172.21.15.160:0/683085996 conn(0x7f9bec100fb0 msgr2=0x7f9bec1088f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:22.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:21.872+0000 7f9bf3124640 1 -- 172.21.15.160:0/683085996 shutdown_connections 2024-09-16T08:40:22.497 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:21.872+0000 7f9bf3124640 1 -- 172.21.15.160:0/683085996 wait complete. 2024-09-16T08:40:22.497 INFO:teuthology.orchestra.run.smithi160.stdout:mgr epoch 5 is available 2024-09-16T08:40:22.497 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying orchestrator module is enabled... 2024-09-16T08:40:23.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:23 smithi160 ceph-mon[30137]: [16/Sep/2024:08:40:22] ENGINE Bus STARTING 2024-09-16T08:40:23.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:23 smithi160 ceph-mon[30137]: [16/Sep/2024:08:40:22] ENGINE Serving on https://172.21.15.160:7150 2024-09-16T08:40:23.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:23 smithi160 ceph-mon[30137]: [16/Sep/2024:08:40:22] ENGINE Error in HTTPServer.serve 2024-09-16T08:40:23.447 INFO:journalctl@ceph.mon.a.smithi160.stdout: Traceback (most recent call last): 2024-09-16T08:40:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:40:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:40:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:40:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: self._run(expiration_interval) 2024-09-16T08:40:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:40:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:40:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:40:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:40:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:40:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: s = self.context.wrap_socket( 2024-09-16T08:40:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:40:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: return self.sslsocket_class._create( 2024-09-16T08:40:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:40:23.449 INFO:journalctl@ceph.mon.a.smithi160.stdout: self.do_handshake() 2024-09-16T08:40:23.449 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:40:23.449 INFO:journalctl@ceph.mon.a.smithi160.stdout: self._sslobj.do_handshake() 2024-09-16T08:40:23.449 INFO:journalctl@ceph.mon.a.smithi160.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:40:23.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:23 smithi160 ceph-mon[30137]: [16/Sep/2024:08:40:22] ENGINE Serving on http://172.21.15.160:8765 2024-09-16T08:40:23.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:23 smithi160 ceph-mon[30137]: [16/Sep/2024:08:40:22] ENGINE Bus STARTED 2024-09-16T08:40:23.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:23 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:40:23.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:23 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/743902388' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-16T08:40:23.904 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.875+0000 7f300bb4e640 1 Processor -- start 2024-09-16T08:40:23.905 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.875+0000 7f300bb4e640 1 -- start start 2024-09-16T08:40:23.905 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.876+0000 7f300bb4e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f3004109ad0 con 0x7f3004105650 2024-09-16T08:40:23.905 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.876+0000 7f30098ea640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f3004105650 0x7f3004105a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36400/0 (socket says 172.21.15.160:36400) 2024-09-16T08:40:23.905 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.876+0000 7f30098ea640 1 -- 172.21.15.160:0/40767750 learned_addr learned my addr 172.21.15.160:0/40767750 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:23.905 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.876+0000 7f30088e8640 1 -- 172.21.15.160:0/40767750 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2372535625 0 0) 0x7f3004109ad0 con 0x7f3004105650 2024-09-16T08:40:23.905 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.876+0000 7f30088e8640 1 -- 172.21.15.160:0/40767750 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fec003520 con 0x7f3004105650 2024-09-16T08:40:23.905 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.876+0000 7f30088e8640 1 -- 172.21.15.160:0/40767750 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 791974744 0 0) 0x7f2fec003520 con 0x7f3004105650 2024-09-16T08:40:23.905 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.876+0000 7f30088e8640 1 -- 172.21.15.160:0/40767750 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300410acb0 con 0x7f3004105650 2024-09-16T08:40:23.905 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.876+0000 7f30088e8640 1 -- 172.21.15.160:0/40767750 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f2ffc001600 con 0x7f3004105650 2024-09-16T08:40:23.905 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.876+0000 7f30088e8640 1 -- 172.21.15.160:0/40767750 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f2ffc001cf0 con 0x7f3004105650 2024-09-16T08:40:23.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.876+0000 7f30088e8640 1 -- 172.21.15.160:0/40767750 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f2ffc0058d0 con 0x7f3004105650 2024-09-16T08:40:23.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.877+0000 7f300bb4e640 1 -- 172.21.15.160:0/40767750 >> v1:172.21.15.160:6789/0 conn(0x7f3004105650 legacy=0x7f3004105a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:23.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.877+0000 7f300bb4e640 1 -- 172.21.15.160:0/40767750 shutdown_connections 2024-09-16T08:40:23.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.877+0000 7f300bb4e640 1 -- 172.21.15.160:0/40767750 >> 172.21.15.160:0/40767750 conn(0x7f3004100f80 msgr2=0x7f30041033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:23.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.877+0000 7f300bb4e640 1 -- 172.21.15.160:0/40767750 shutdown_connections 2024-09-16T08:40:23.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.877+0000 7f300bb4e640 1 -- 172.21.15.160:0/40767750 wait complete. 2024-09-16T08:40:23.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.877+0000 7f300bb4e640 1 Processor -- start 2024-09-16T08:40:23.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.877+0000 7f300bb4e640 1 -- start start 2024-09-16T08:40:23.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.877+0000 7f300bb4e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f300419e010 con 0x7f3004105650 2024-09-16T08:40:23.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.878+0000 7f30098ea640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f3004105650 0x7f30040797e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36412/0 (socket says 172.21.15.160:36412) 2024-09-16T08:40:23.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.878+0000 7f30098ea640 1 -- 172.21.15.160:0/743902388 learned_addr learned my addr 172.21.15.160:0/743902388 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:23.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.878+0000 7f3002ffd640 1 -- 172.21.15.160:0/743902388 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1434115516 0 0) 0x7f300419e010 con 0x7f3004105650 2024-09-16T08:40:23.906 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.878+0000 7f3002ffd640 1 -- 172.21.15.160:0/743902388 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fec005570 con 0x7f3004105650 2024-09-16T08:40:23.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.878+0000 7f3002ffd640 1 -- 172.21.15.160:0/743902388 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 101725788 0 0) 0x7f2fec005570 con 0x7f3004105650 2024-09-16T08:40:23.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.878+0000 7f3002ffd640 1 -- 172.21.15.160:0/743902388 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f300419e010 con 0x7f3004105650 2024-09-16T08:40:23.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.878+0000 7f3002ffd640 1 -- 172.21.15.160:0/743902388 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f2ffc001080 con 0x7f3004105650 2024-09-16T08:40:23.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.878+0000 7f3002ffd640 1 -- 172.21.15.160:0/743902388 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2787439918 0 0) 0x7f300419e010 con 0x7f3004105650 2024-09-16T08:40:23.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.878+0000 7f3002ffd640 1 -- 172.21.15.160:0/743902388 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300419f1f0 con 0x7f3004105650 2024-09-16T08:40:23.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.878+0000 7f300bb4e640 1 -- 172.21.15.160:0/743902388 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f300419e240 con 0x7f3004105650 2024-09-16T08:40:23.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.878+0000 7f300bb4e640 1 -- 172.21.15.160:0/743902388 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f300419e780 con 0x7f3004105650 2024-09-16T08:40:23.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.878+0000 7f3002ffd640 1 -- 172.21.15.160:0/743902388 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f2ffc001820 con 0x7f3004105650 2024-09-16T08:40:23.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.878+0000 7f3002ffd640 1 -- 172.21.15.160:0/743902388 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f2ffc006030 con 0x7f3004105650 2024-09-16T08:40:23.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.879+0000 7f3002ffd640 1 -- 172.21.15.160:0/743902388 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 7) ==== 51624+0+0 (unknown 2115241582 0 0) 0x7f2ffc012b80 con 0x7f3004105650 2024-09-16T08:40:23.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.879+0000 7f300bb4e640 1 -- 172.21.15.160:0/743902388 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff00050b0 con 0x7f3004105650 2024-09-16T08:40:23.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.879+0000 7f3002ffd640 1 -- 172.21.15.160:0/743902388 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 760617087 0 0) 0x7f2ffc04f450 con 0x7f3004105650 2024-09-16T08:40:23.907 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:22.884+0000 7f3002ffd640 1 -- 172.21.15.160:0/743902388 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f2ffc004a20 con 0x7f3004105650 2024-09-16T08:40:23.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:23.108+0000 7f300bb4e640 1 -- 172.21.15.160:0/743902388 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "orchestrator"} v 0) -- 0x7f2ff0005c70 con 0x7f3004105650 2024-09-16T08:40:23.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:23.276+0000 7f3002ffd640 1 -- 172.21.15.160:0/743902388 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "orchestrator"}]=0 module 'orchestrator' is already enabled (always-on) v8) ==== 143+0+0 (unknown 114785661 0 0) 0x7f2ffc016320 con 0x7f3004105650 2024-09-16T08:40:23.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-16T08:40:23.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:23.277+0000 7f3002ffd640 1 -- 172.21.15.160:0/743902388 <== mon.0 v1:172.21.15.160:6789/0 11 ==== mgrmap(e 8) ==== 51730+0+0 (unknown 4165010166 0 0) 0x7f2ffc04dfc0 con 0x7f3004105650 2024-09-16T08:40:23.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:23.278+0000 7f300bb4e640 1 -- 172.21.15.160:0/743902388 >> v1:172.21.15.160:6800/3425583652 conn(0x7f2fec040750 legacy=0x7f2fec042c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:23.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:23.278+0000 7f300bb4e640 1 -- 172.21.15.160:0/743902388 >> v1:172.21.15.160:6789/0 conn(0x7f3004105650 legacy=0x7f30040797e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:23.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:23.279+0000 7f300bb4e640 1 -- 172.21.15.160:0/743902388 shutdown_connections 2024-09-16T08:40:23.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:23.279+0000 7f300bb4e640 1 -- 172.21.15.160:0/743902388 >> 172.21.15.160:0/743902388 conn(0x7f3004100f80 msgr2=0x7f30041033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:23.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:23.279+0000 7f300bb4e640 1 -- 172.21.15.160:0/743902388 shutdown_connections 2024-09-16T08:40:23.908 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:23.279+0000 7f300bb4e640 1 -- 172.21.15.160:0/743902388 wait complete. 2024-09-16T08:40:23.908 INFO:teuthology.orchestra.run.smithi160.stdout:Setting orchestrator backend to cephadm... 2024-09-16T08:40:24.541 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/743902388' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-16T08:40:24.541 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:24 smithi160 ceph-mon[30137]: mgrmap e8: y(active, since 2s) 2024-09-16T08:40:25.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.340+0000 7f3304f72640 1 Processor -- start 2024-09-16T08:40:25.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.340+0000 7f3304f72640 1 -- start start 2024-09-16T08:40:25.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.340+0000 7f3304f72640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f330010aad0 con 0x7f330007a400 2024-09-16T08:40:25.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.340+0000 7f32fe59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f330007a400 0x7f3300109320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36418/0 (socket says 172.21.15.160:36418) 2024-09-16T08:40:25.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.340+0000 7f32fe59c640 1 -- 172.21.15.160:0/253065460 learned_addr learned my addr 172.21.15.160:0/253065460 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:25.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.341+0000 7f32fd59a640 1 -- 172.21.15.160:0/253065460 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 774326800 0 0) 0x7f330010aad0 con 0x7f330007a400 2024-09-16T08:40:25.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.341+0000 7f32fd59a640 1 -- 172.21.15.160:0/253065460 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32e8003520 con 0x7f330007a400 2024-09-16T08:40:25.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.341+0000 7f32fd59a640 1 -- 172.21.15.160:0/253065460 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3967187869 0 0) 0x7f32e8003520 con 0x7f330007a400 2024-09-16T08:40:25.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.341+0000 7f32fd59a640 1 -- 172.21.15.160:0/253065460 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330010bcb0 con 0x7f330007a400 2024-09-16T08:40:25.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.341+0000 7f32fd59a640 1 -- 172.21.15.160:0/253065460 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f32f0001600 con 0x7f330007a400 2024-09-16T08:40:25.127 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.341+0000 7f32fd59a640 1 -- 172.21.15.160:0/253065460 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f32f0001cf0 con 0x7f330007a400 2024-09-16T08:40:25.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.341+0000 7f32fd59a640 1 -- 172.21.15.160:0/253065460 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f32f00058d0 con 0x7f330007a400 2024-09-16T08:40:25.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.342+0000 7f3304f72640 1 -- 172.21.15.160:0/253065460 >> v1:172.21.15.160:6789/0 conn(0x7f330007a400 legacy=0x7f3300109320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:25.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.342+0000 7f3304f72640 1 -- 172.21.15.160:0/253065460 shutdown_connections 2024-09-16T08:40:25.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.342+0000 7f3304f72640 1 -- 172.21.15.160:0/253065460 >> 172.21.15.160:0/253065460 conn(0x7f3300100f90 msgr2=0x7f33001033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:25.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.342+0000 7f3304f72640 1 -- 172.21.15.160:0/253065460 shutdown_connections 2024-09-16T08:40:25.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.342+0000 7f3304f72640 1 -- 172.21.15.160:0/253065460 wait complete. 2024-09-16T08:40:25.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.342+0000 7f3304f72640 1 Processor -- start 2024-09-16T08:40:25.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.342+0000 7f3304f72640 1 -- start start 2024-09-16T08:40:25.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.343+0000 7f3304f72640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f33001070d0 con 0x7f330007a400 2024-09-16T08:40:25.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.343+0000 7f32fe59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f330007a400 0x7f3300106970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36430/0 (socket says 172.21.15.160:36430) 2024-09-16T08:40:25.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.343+0000 7f32fe59c640 1 -- 172.21.15.160:0/1372857081 learned_addr learned my addr 172.21.15.160:0/1372857081 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:25.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.343+0000 7f32ef7fe640 1 -- 172.21.15.160:0/1372857081 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1475080878 0 0) 0x7f33001070d0 con 0x7f330007a400 2024-09-16T08:40:25.128 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.343+0000 7f32ef7fe640 1 -- 172.21.15.160:0/1372857081 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32e8005570 con 0x7f330007a400 2024-09-16T08:40:25.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.343+0000 7f32ef7fe640 1 -- 172.21.15.160:0/1372857081 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2273455831 0 0) 0x7f32e8005570 con 0x7f330007a400 2024-09-16T08:40:25.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.343+0000 7f32ef7fe640 1 -- 172.21.15.160:0/1372857081 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f33001070d0 con 0x7f330007a400 2024-09-16T08:40:25.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.343+0000 7f32ef7fe640 1 -- 172.21.15.160:0/1372857081 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f32f0001080 con 0x7f330007a400 2024-09-16T08:40:25.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.343+0000 7f32ef7fe640 1 -- 172.21.15.160:0/1372857081 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2908351252 0 0) 0x7f33001070d0 con 0x7f330007a400 2024-09-16T08:40:25.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.343+0000 7f32ef7fe640 1 -- 172.21.15.160:0/1372857081 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33001a12d0 con 0x7f330007a400 2024-09-16T08:40:25.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.343+0000 7f3304f72640 1 -- 172.21.15.160:0/1372857081 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f33001a0320 con 0x7f330007a400 2024-09-16T08:40:25.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.343+0000 7f3304f72640 1 -- 172.21.15.160:0/1372857081 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f33001a0860 con 0x7f330007a400 2024-09-16T08:40:25.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.343+0000 7f32ef7fe640 1 -- 172.21.15.160:0/1372857081 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f32f0001820 con 0x7f330007a400 2024-09-16T08:40:25.129 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.343+0000 7f32ef7fe640 1 -- 172.21.15.160:0/1372857081 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f32f0006030 con 0x7f330007a400 2024-09-16T08:40:25.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.344+0000 7f32ef7fe640 1 -- 172.21.15.160:0/1372857081 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 8) ==== 51730+0+0 (unknown 4165010166 0 0) 0x7f32f0012bf0 con 0x7f330007a400 2024-09-16T08:40:25.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.344+0000 7f3304f72640 1 -- 172.21.15.160:0/1372857081 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e00050b0 con 0x7f330007a400 2024-09-16T08:40:25.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.344+0000 7f32ef7fe640 1 -- 172.21.15.160:0/1372857081 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 760617087 0 0) 0x7f32f004f6f0 con 0x7f330007a400 2024-09-16T08:40:25.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.349+0000 7f32ef7fe640 1 -- 172.21.15.160:0/1372857081 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f32f0018e20 con 0x7f330007a400 2024-09-16T08:40:25.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.540+0000 7f3304f72640 1 -- 172.21.15.160:0/1372857081 --> v1:172.21.15.160:6800/3425583652 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f32e0000bc0 con 0x7f32e8040860 2024-09-16T08:40:25.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.545+0000 7f32ef7fe640 1 -- 172.21.15.160:0/1372857081 <== mgr.14116 v1:172.21.15.160:6800/3425583652 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (unknown 0 0 0) 0x7f32e0000bc0 con 0x7f32e8040860 2024-09-16T08:40:25.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.547+0000 7f3304f72640 1 -- 172.21.15.160:0/1372857081 >> v1:172.21.15.160:6800/3425583652 conn(0x7f32e8040860 legacy=0x7f32e8042d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:25.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.547+0000 7f3304f72640 1 -- 172.21.15.160:0/1372857081 >> v1:172.21.15.160:6789/0 conn(0x7f330007a400 legacy=0x7f3300106970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:25.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.547+0000 7f3304f72640 1 -- 172.21.15.160:0/1372857081 shutdown_connections 2024-09-16T08:40:25.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.547+0000 7f3304f72640 1 -- 172.21.15.160:0/1372857081 >> 172.21.15.160:0/1372857081 conn(0x7f3300100f90 msgr2=0x7f3300108900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:25.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.547+0000 7f3304f72640 1 -- 172.21.15.160:0/1372857081 shutdown_connections 2024-09-16T08:40:25.130 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:24.547+0000 7f3304f72640 1 -- 172.21.15.160:0/1372857081 wait complete. 2024-09-16T08:40:25.755 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:25 smithi160 ceph-mon[30137]: from='client.14130 v1:172.21.15.160:0/1372857081' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:40:25.755 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:25 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' 2024-09-16T08:40:25.755 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:25 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:40:26.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-16T08:40:26.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.569+0000 7f59dc8ad640 1 Processor -- start 2024-09-16T08:40:26.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.569+0000 7f59dc8ad640 1 -- start start 2024-09-16T08:40:26.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.569+0000 7f59dc8ad640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f59d810abe0 con 0x7f59d8106760 2024-09-16T08:40:26.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.569+0000 7f59d659c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f59d8106760 0x7f59d8106b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36432/0 (socket says 172.21.15.160:36432) 2024-09-16T08:40:26.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.569+0000 7f59d659c640 1 -- 172.21.15.160:0/3345202076 learned_addr learned my addr 172.21.15.160:0/3345202076 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:26.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.569+0000 7f59d5d9b640 1 -- 172.21.15.160:0/3345202076 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2837357030 0 0) 0x7f59d810abe0 con 0x7f59d8106760 2024-09-16T08:40:26.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.569+0000 7f59d5d9b640 1 -- 172.21.15.160:0/3345202076 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59c4003520 con 0x7f59d8106760 2024-09-16T08:40:26.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.570+0000 7f59d5d9b640 1 -- 172.21.15.160:0/3345202076 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4194198337 0 0) 0x7f59c4003520 con 0x7f59d8106760 2024-09-16T08:40:26.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.570+0000 7f59d5d9b640 1 -- 172.21.15.160:0/3345202076 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d810bdc0 con 0x7f59d8106760 2024-09-16T08:40:26.325 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.570+0000 7f59d5d9b640 1 -- 172.21.15.160:0/3345202076 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f59c8001600 con 0x7f59d8106760 2024-09-16T08:40:26.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.570+0000 7f59d5d9b640 1 -- 172.21.15.160:0/3345202076 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f59c8001cf0 con 0x7f59d8106760 2024-09-16T08:40:26.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.570+0000 7f59d5d9b640 1 -- 172.21.15.160:0/3345202076 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f59c80058d0 con 0x7f59d8106760 2024-09-16T08:40:26.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.570+0000 7f59dc8ad640 1 -- 172.21.15.160:0/3345202076 >> v1:172.21.15.160:6789/0 conn(0x7f59d8106760 legacy=0x7f59d8106b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:26.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.570+0000 7f59dc8ad640 1 -- 172.21.15.160:0/3345202076 shutdown_connections 2024-09-16T08:40:26.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.570+0000 7f59dc8ad640 1 -- 172.21.15.160:0/3345202076 >> 172.21.15.160:0/3345202076 conn(0x7f59d8102070 msgr2=0x7f59d8104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:26.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.570+0000 7f59dc8ad640 1 -- 172.21.15.160:0/3345202076 shutdown_connections 2024-09-16T08:40:26.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.570+0000 7f59dc8ad640 1 -- 172.21.15.160:0/3345202076 wait complete. 2024-09-16T08:40:26.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.571+0000 7f59dc8ad640 1 Processor -- start 2024-09-16T08:40:26.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.571+0000 7f59dc8ad640 1 -- start start 2024-09-16T08:40:26.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.571+0000 7f59dc8ad640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f59d819aae0 con 0x7f59d8106760 2024-09-16T08:40:26.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.571+0000 7f59d659c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f59d8106760 0x7f59d819a380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36436/0 (socket says 172.21.15.160:36436) 2024-09-16T08:40:26.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.571+0000 7f59d659c640 1 -- 172.21.15.160:0/856054533 learned_addr learned my addr 172.21.15.160:0/856054533 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:26.326 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.571+0000 7f59cf7fe640 1 -- 172.21.15.160:0/856054533 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3854482763 0 0) 0x7f59d819aae0 con 0x7f59d8106760 2024-09-16T08:40:26.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.572+0000 7f59cf7fe640 1 -- 172.21.15.160:0/856054533 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59c4005570 con 0x7f59d8106760 2024-09-16T08:40:26.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.572+0000 7f59cf7fe640 1 -- 172.21.15.160:0/856054533 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2966665397 0 0) 0x7f59c4005570 con 0x7f59d8106760 2024-09-16T08:40:26.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.572+0000 7f59cf7fe640 1 -- 172.21.15.160:0/856054533 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f59d819aae0 con 0x7f59d8106760 2024-09-16T08:40:26.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.572+0000 7f59cf7fe640 1 -- 172.21.15.160:0/856054533 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f59c80047e0 con 0x7f59d8106760 2024-09-16T08:40:26.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.572+0000 7f59cf7fe640 1 -- 172.21.15.160:0/856054533 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1544068642 0 0) 0x7f59d819aae0 con 0x7f59d8106760 2024-09-16T08:40:26.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.572+0000 7f59cf7fe640 1 -- 172.21.15.160:0/856054533 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d819acd0 con 0x7f59d8106760 2024-09-16T08:40:26.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.572+0000 7f59dc8ad640 1 -- 172.21.15.160:0/856054533 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f59d819e5b0 con 0x7f59d8106760 2024-09-16T08:40:26.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.572+0000 7f59dc8ad640 1 -- 172.21.15.160:0/856054533 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f59d819e9d0 con 0x7f59d8106760 2024-09-16T08:40:26.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.572+0000 7f59cf7fe640 1 -- 172.21.15.160:0/856054533 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f59c8001bd0 con 0x7f59d8106760 2024-09-16T08:40:26.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.572+0000 7f59cf7fe640 1 -- 172.21.15.160:0/856054533 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f59c8005f50 con 0x7f59d8106760 2024-09-16T08:40:26.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.573+0000 7f59dc8ad640 1 -- 172.21.15.160:0/856054533 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b80050b0 con 0x7f59d8106760 2024-09-16T08:40:26.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.573+0000 7f59cf7fe640 1 -- 172.21.15.160:0/856054533 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 8) ==== 51730+0+0 (unknown 4165010166 0 0) 0x7f59c8012b90 con 0x7f59d8106760 2024-09-16T08:40:26.327 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.573+0000 7f59cf7fe640 1 -- 172.21.15.160:0/856054533 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 760617087 0 0) 0x7f59c804f780 con 0x7f59d8106760 2024-09-16T08:40:26.328 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.578+0000 7f59cf7fe640 1 -- 172.21.15.160:0/856054533 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f59c8004b90 con 0x7f59d8106760 2024-09-16T08:40:26.328 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.754+0000 7f59dc8ad640 1 -- 172.21.15.160:0/856054533 --> v1:172.21.15.160:6800/3425583652 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f59b8000bc0 con 0x7f59c40408a0 2024-09-16T08:40:26.328 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.755+0000 7f59cf7fe640 1 -- 172.21.15.160:0/856054533 <== mgr.14116 v1:172.21.15.160:6800/3425583652 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (unknown 0 0 2070689548) 0x7f59b8000bc0 con 0x7f59c40408a0 2024-09-16T08:40:26.328 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.757+0000 7f59dc8ad640 1 -- 172.21.15.160:0/856054533 >> v1:172.21.15.160:6800/3425583652 conn(0x7f59c40408a0 legacy=0x7f59c4042d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:26.328 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.757+0000 7f59dc8ad640 1 -- 172.21.15.160:0/856054533 >> v1:172.21.15.160:6789/0 conn(0x7f59d8106760 legacy=0x7f59d819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:26.328 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.757+0000 7f59dc8ad640 1 -- 172.21.15.160:0/856054533 shutdown_connections 2024-09-16T08:40:26.328 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.757+0000 7f59dc8ad640 1 -- 172.21.15.160:0/856054533 >> 172.21.15.160:0/856054533 conn(0x7f59d8102070 msgr2=0x7f59d8104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:26.328 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.757+0000 7f59dc8ad640 1 -- 172.21.15.160:0/856054533 shutdown_connections 2024-09-16T08:40:26.328 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:25.757+0000 7f59dc8ad640 1 -- 172.21.15.160:0/856054533 wait complete. 2024-09-16T08:40:26.328 INFO:teuthology.orchestra.run.smithi160.stdout:Generating ssh key... 2024-09-16T08:40:26.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:26 smithi160 ceph-mon[30137]: from='client.14132 v1:172.21.15.160:0/856054533' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:40:27.197 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: Generating public/private rsa key pair. 2024-09-16T08:40:27.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: Your identification has been saved in /tmp/tmp1trubmn_/key 2024-09-16T08:40:27.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: Your public key has been saved in /tmp/tmp1trubmn_/key.pub 2024-09-16T08:40:27.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: The key fingerprint is: 2024-09-16T08:40:27.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: SHA256:3SLdMO/lR1DApIGcZpyUn4MCOHwpo5UvyuE4c05vNr0 ceph-27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:40:27.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: The key's randomart image is: 2024-09-16T08:40:27.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: +---[RSA 3072]----+ 2024-09-16T08:40:27.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: | . o . +.=.oo..| 2024-09-16T08:40:27.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: | O + X o.. | 2024-09-16T08:40:27.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: | o * . ooo... | 2024-09-16T08:40:27.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: | o . . .o.*+ . | 2024-09-16T08:40:27.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: | + o . S.+ +.. .| 2024-09-16T08:40:27.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: |+ * . o o . | 2024-09-16T08:40:27.199 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: | * . . . . .| 2024-09-16T08:40:27.199 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: | . = . . | 2024-09-16T08:40:27.199 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: | o .E. | 2024-09-16T08:40:27.199 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: +----[SHA256]-----+ 2024-09-16T08:40:27.640 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.756+0000 7f8989942640 1 Processor -- start 2024-09-16T08:40:27.640 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.756+0000 7f8989942640 1 -- start start 2024-09-16T08:40:27.640 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.756+0000 7f8989942640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f898410ab90 con 0x7f8984106710 2024-09-16T08:40:27.640 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.756+0000 7f8982ffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f8984106710 0x7f8984106b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36440/0 (socket says 172.21.15.160:36440) 2024-09-16T08:40:27.640 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.756+0000 7f8982ffd640 1 -- 172.21.15.160:0/1162933389 learned_addr learned my addr 172.21.15.160:0/1162933389 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:27.640 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.757+0000 7f8981ffb640 1 -- 172.21.15.160:0/1162933389 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1519429945 0 0) 0x7f898410ab90 con 0x7f8984106710 2024-09-16T08:40:27.640 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.757+0000 7f8981ffb640 1 -- 172.21.15.160:0/1162933389 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8970003520 con 0x7f8984106710 2024-09-16T08:40:27.640 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.757+0000 7f8981ffb640 1 -- 172.21.15.160:0/1162933389 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 413418449 0 0) 0x7f8970003520 con 0x7f8984106710 2024-09-16T08:40:27.641 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.757+0000 7f8981ffb640 1 -- 172.21.15.160:0/1162933389 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898410bd70 con 0x7f8984106710 2024-09-16T08:40:27.641 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.757+0000 7f8981ffb640 1 -- 172.21.15.160:0/1162933389 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f8974001980 con 0x7f8984106710 2024-09-16T08:40:27.641 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.757+0000 7f8981ffb640 1 -- 172.21.15.160:0/1162933389 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f8974002070 con 0x7f8984106710 2024-09-16T08:40:27.641 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.757+0000 7f8981ffb640 1 -- 172.21.15.160:0/1162933389 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f8974005c50 con 0x7f8984106710 2024-09-16T08:40:27.641 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.757+0000 7f8989942640 1 -- 172.21.15.160:0/1162933389 >> v1:172.21.15.160:6789/0 conn(0x7f8984106710 legacy=0x7f8984106b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:27.641 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.758+0000 7f8989942640 1 -- 172.21.15.160:0/1162933389 shutdown_connections 2024-09-16T08:40:27.641 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.758+0000 7f8989942640 1 -- 172.21.15.160:0/1162933389 >> 172.21.15.160:0/1162933389 conn(0x7f8984102000 msgr2=0x7f8984104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:27.641 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.758+0000 7f8989942640 1 -- 172.21.15.160:0/1162933389 shutdown_connections 2024-09-16T08:40:27.641 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.758+0000 7f8989942640 1 -- 172.21.15.160:0/1162933389 wait complete. 2024-09-16T08:40:27.641 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.758+0000 7f8989942640 1 Processor -- start 2024-09-16T08:40:27.642 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.758+0000 7f8989942640 1 -- start start 2024-09-16T08:40:27.642 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.758+0000 7f8989942640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f898419aa30 con 0x7f8984106710 2024-09-16T08:40:27.642 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.759+0000 7f8982ffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f8984106710 0x7f898419a2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36452/0 (socket says 172.21.15.160:36452) 2024-09-16T08:40:27.642 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.759+0000 7f8982ffd640 1 -- 172.21.15.160:0/2762220201 learned_addr learned my addr 172.21.15.160:0/2762220201 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:27.642 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.759+0000 7f8988940640 1 -- 172.21.15.160:0/2762220201 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3556641638 0 0) 0x7f898419aa30 con 0x7f8984106710 2024-09-16T08:40:27.642 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.759+0000 7f8988940640 1 -- 172.21.15.160:0/2762220201 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8970005570 con 0x7f8984106710 2024-09-16T08:40:27.642 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.759+0000 7f8988940640 1 -- 172.21.15.160:0/2762220201 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3545905502 0 0) 0x7f8970005570 con 0x7f8984106710 2024-09-16T08:40:27.642 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.759+0000 7f8988940640 1 -- 172.21.15.160:0/2762220201 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f898419aa30 con 0x7f8984106710 2024-09-16T08:40:27.643 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.759+0000 7f8988940640 1 -- 172.21.15.160:0/2762220201 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f8974001400 con 0x7f8984106710 2024-09-16T08:40:27.643 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.759+0000 7f8988940640 1 -- 172.21.15.160:0/2762220201 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 641922641 0 0) 0x7f898419aa30 con 0x7f8984106710 2024-09-16T08:40:27.643 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.759+0000 7f8988940640 1 -- 172.21.15.160:0/2762220201 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898419ac20 con 0x7f8984106710 2024-09-16T08:40:27.643 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.759+0000 7f8989942640 1 -- 172.21.15.160:0/2762220201 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f898419e500 con 0x7f8984106710 2024-09-16T08:40:27.643 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.759+0000 7f8989942640 1 -- 172.21.15.160:0/2762220201 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f898419e920 con 0x7f8984106710 2024-09-16T08:40:27.643 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.759+0000 7f8988940640 1 -- 172.21.15.160:0/2762220201 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f8974001ba0 con 0x7f8984106710 2024-09-16T08:40:27.643 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.759+0000 7f8988940640 1 -- 172.21.15.160:0/2762220201 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f8974002070 con 0x7f8984106710 2024-09-16T08:40:27.643 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.760+0000 7f8989942640 1 -- 172.21.15.160:0/2762220201 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89640050b0 con 0x7f8984106710 2024-09-16T08:40:27.643 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.760+0000 7f8988940640 1 -- 172.21.15.160:0/2762220201 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 8) ==== 51730+0+0 (unknown 4165010166 0 0) 0x7f8974013000 con 0x7f8984106710 2024-09-16T08:40:27.643 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.761+0000 7f8988940640 1 -- 172.21.15.160:0/2762220201 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 760617087 0 0) 0x7f897404fcf0 con 0x7f8984106710 2024-09-16T08:40:27.643 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.765+0000 7f8988940640 1 -- 172.21.15.160:0/2762220201 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f8974019280 con 0x7f8984106710 2024-09-16T08:40:27.643 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:26.941+0000 7f8989942640 1 -- 172.21.15.160:0/2762220201 --> v1:172.21.15.160:6800/3425583652 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f8964000bc0 con 0x7f89700409c0 2024-09-16T08:40:27.643 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:27.105+0000 7f8988940640 1 -- 172.21.15.160:0/2762220201 <== mgr.14116 v1:172.21.15.160:6800/3425583652 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (unknown 0 0 0) 0x7f8964000bc0 con 0x7f89700409c0 2024-09-16T08:40:27.644 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:27.108+0000 7f8989942640 1 -- 172.21.15.160:0/2762220201 >> v1:172.21.15.160:6800/3425583652 conn(0x7f89700409c0 legacy=0x7f8970042e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:27.644 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:27.108+0000 7f8989942640 1 -- 172.21.15.160:0/2762220201 >> v1:172.21.15.160:6789/0 conn(0x7f8984106710 legacy=0x7f898419a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:27.644 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:27.108+0000 7f8989942640 1 -- 172.21.15.160:0/2762220201 shutdown_connections 2024-09-16T08:40:27.644 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:27.108+0000 7f8989942640 1 -- 172.21.15.160:0/2762220201 >> 172.21.15.160:0/2762220201 conn(0x7f8984102000 msgr2=0x7f89841043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:27.644 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:27.108+0000 7f8989942640 1 -- 172.21.15.160:0/2762220201 shutdown_connections 2024-09-16T08:40:27.644 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:27.108+0000 7f8989942640 1 -- 172.21.15.160:0/2762220201 wait complete. 2024-09-16T08:40:28.258 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:28 smithi160 ceph-mon[30137]: from='client.14134 v1:172.21.15.160:0/2762220201' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:40:28.259 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:28 smithi160 ceph-mon[30137]: Generating ssh key... 2024-09-16T08:40:28.259 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:28 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' 2024-09-16T08:40:28.259 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:28 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' 2024-09-16T08:40:28.850 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFo1RwGOUA+EXBp0a7t2lR/doCyOZPDvvAke0ImskAGdRd1Cok3ia1ioVV+lFtFFpS+n+nQ5ot1KbAMGCeMKQHcUTlrRjDzhUqRIj7XUo7iarzcIfiC2bEgitS0oA0EG5uy8JeiWgfSre+KtFjw7BewLwzKfD3XavrMAgF49dKmqctfxVV3maghdfJCjLP4//CL1NIe8BR8LdLmT9SVwSSgLCDnn+1lrZ1VsD6jg93naVEoQnO0Wfss4T6Tef1YCo48sV0/OY3Hqh1kdDilGjOrMxzwnwkzddJqnZLSaDsQYX+lSKFCgPpNnJMiVDJxw+tHT0RwptIU/Q/7mYYIjwJERlUJUfgNbfAsUGJw9bUg8CGOU0j2Bs/hoRt81xyDxV+7pkjMn1owK1ygXeZAGf97nzcf9xo/6AVt+IFQQHA2oh+4v6Dz7tsuXbktO0Lik6RgRUQHLfe0ThW6ganch7lFOtQBGclNTmPc9A3/nTKGYdXQzLYFnP6sMFnh80RU5M= ceph-27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:40:28.850 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.072+0000 7fd78213f640 1 Processor -- start 2024-09-16T08:40:28.851 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.072+0000 7fd78213f640 1 -- start start 2024-09-16T08:40:28.851 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.072+0000 7fd78213f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd77c07b1f0 con 0x7fd77c07c480 2024-09-16T08:40:28.851 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.072+0000 7fd77b7fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd77c07c480 0x7fd77c07aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:52828/0 (socket says 172.21.15.160:52828) 2024-09-16T08:40:28.851 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.072+0000 7fd77b7fe640 1 -- 172.21.15.160:0/3705061208 learned_addr learned my addr 172.21.15.160:0/3705061208 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:28.851 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.072+0000 7fd77affd640 1 -- 172.21.15.160:0/3705061208 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1443185715 0 0) 0x7fd77c07b1f0 con 0x7fd77c07c480 2024-09-16T08:40:28.851 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.072+0000 7fd77affd640 1 -- 172.21.15.160:0/3705061208 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd768003520 con 0x7fd77c07c480 2024-09-16T08:40:28.851 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.073+0000 7fd77affd640 1 -- 172.21.15.160:0/3705061208 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2956121986 0 0) 0x7fd768003520 con 0x7fd77c07c480 2024-09-16T08:40:28.851 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.073+0000 7fd77affd640 1 -- 172.21.15.160:0/3705061208 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77c07b3c0 con 0x7fd77c07c480 2024-09-16T08:40:28.851 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.073+0000 7fd77affd640 1 -- 172.21.15.160:0/3705061208 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fd76c001600 con 0x7fd77c07c480 2024-09-16T08:40:28.851 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.073+0000 7fd77affd640 1 -- 172.21.15.160:0/3705061208 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7fd76c001cf0 con 0x7fd77c07c480 2024-09-16T08:40:28.851 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.073+0000 7fd77affd640 1 -- 172.21.15.160:0/3705061208 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fd76c0058d0 con 0x7fd77c07c480 2024-09-16T08:40:28.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.073+0000 7fd78213f640 1 -- 172.21.15.160:0/3705061208 >> v1:172.21.15.160:6789/0 conn(0x7fd77c07c480 legacy=0x7fd77c07aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:28.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.073+0000 7fd78213f640 1 -- 172.21.15.160:0/3705061208 shutdown_connections 2024-09-16T08:40:28.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.073+0000 7fd78213f640 1 -- 172.21.15.160:0/3705061208 >> 172.21.15.160:0/3705061208 conn(0x7fd77c102070 msgr2=0x7fd77c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:28.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.073+0000 7fd78213f640 1 -- 172.21.15.160:0/3705061208 shutdown_connections 2024-09-16T08:40:28.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.073+0000 7fd78213f640 1 -- 172.21.15.160:0/3705061208 wait complete. 2024-09-16T08:40:28.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.074+0000 7fd78213f640 1 Processor -- start 2024-09-16T08:40:28.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.074+0000 7fd78213f640 1 -- start start 2024-09-16T08:40:28.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.074+0000 7fd78213f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd77c19cc20 con 0x7fd77c07c480 2024-09-16T08:40:28.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.074+0000 7fd77b7fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd77c07c480 0x7fd77c19c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:52842/0 (socket says 172.21.15.160:52842) 2024-09-16T08:40:28.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.074+0000 7fd77b7fe640 1 -- 172.21.15.160:0/3294071914 learned_addr learned my addr 172.21.15.160:0/3294071914 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:28.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.074+0000 7fd7797fa640 1 -- 172.21.15.160:0/3294071914 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1980087872 0 0) 0x7fd77c19cc20 con 0x7fd77c07c480 2024-09-16T08:40:28.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.074+0000 7fd7797fa640 1 -- 172.21.15.160:0/3294071914 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd768005570 con 0x7fd77c07c480 2024-09-16T08:40:28.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.074+0000 7fd7797fa640 1 -- 172.21.15.160:0/3294071914 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3535942534 0 0) 0x7fd768005570 con 0x7fd77c07c480 2024-09-16T08:40:28.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.075+0000 7fd7797fa640 1 -- 172.21.15.160:0/3294071914 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd77c19cc20 con 0x7fd77c07c480 2024-09-16T08:40:28.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.075+0000 7fd7797fa640 1 -- 172.21.15.160:0/3294071914 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fd76c001080 con 0x7fd77c07c480 2024-09-16T08:40:28.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.075+0000 7fd7797fa640 1 -- 172.21.15.160:0/3294071914 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1630056265 0 0) 0x7fd77c19cc20 con 0x7fd77c07c480 2024-09-16T08:40:28.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.075+0000 7fd7797fa640 1 -- 172.21.15.160:0/3294071914 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77c19ce10 con 0x7fd77c07c480 2024-09-16T08:40:28.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.075+0000 7fd78213f640 1 -- 172.21.15.160:0/3294071914 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd77c1a06f0 con 0x7fd77c07c480 2024-09-16T08:40:28.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.075+0000 7fd78213f640 1 -- 172.21.15.160:0/3294071914 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd77c1a0b10 con 0x7fd77c07c480 2024-09-16T08:40:28.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.075+0000 7fd7797fa640 1 -- 172.21.15.160:0/3294071914 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7fd76c001820 con 0x7fd77c07c480 2024-09-16T08:40:28.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.075+0000 7fd7797fa640 1 -- 172.21.15.160:0/3294071914 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fd76c006030 con 0x7fd77c07c480 2024-09-16T08:40:28.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.076+0000 7fd7797fa640 1 -- 172.21.15.160:0/3294071914 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 8) ==== 51730+0+0 (unknown 4165010166 0 0) 0x7fd76c012bf0 con 0x7fd77c07c480 2024-09-16T08:40:28.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.076+0000 7fd78213f640 1 -- 172.21.15.160:0/3294071914 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd75c0050b0 con 0x7fd77c07c480 2024-09-16T08:40:28.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.076+0000 7fd7797fa640 1 -- 172.21.15.160:0/3294071914 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 760617087 0 0) 0x7fd76c04f910 con 0x7fd77c07c480 2024-09-16T08:40:28.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.081+0000 7fd7797fa640 1 -- 172.21.15.160:0/3294071914 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fd76c018e20 con 0x7fd77c07c480 2024-09-16T08:40:28.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.258+0000 7fd78213f640 1 -- 172.21.15.160:0/3294071914 --> v1:172.21.15.160:6800/3425583652 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fd75c000bc0 con 0x7fd768040860 2024-09-16T08:40:28.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.259+0000 7fd7797fa640 1 -- 172.21.15.160:0/3294071914 <== mgr.14116 v1:172.21.15.160:6800/3425583652 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (unknown 0 0 587516700) 0x7fd75c000bc0 con 0x7fd768040860 2024-09-16T08:40:28.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.261+0000 7fd78213f640 1 -- 172.21.15.160:0/3294071914 >> v1:172.21.15.160:6800/3425583652 conn(0x7fd768040860 legacy=0x7fd768042d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:28.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.261+0000 7fd78213f640 1 -- 172.21.15.160:0/3294071914 >> v1:172.21.15.160:6789/0 conn(0x7fd77c07c480 legacy=0x7fd77c19c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:28.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.261+0000 7fd78213f640 1 -- 172.21.15.160:0/3294071914 shutdown_connections 2024-09-16T08:40:28.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.261+0000 7fd78213f640 1 -- 172.21.15.160:0/3294071914 >> 172.21.15.160:0/3294071914 conn(0x7fd77c102070 msgr2=0x7fd77c10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:28.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.261+0000 7fd78213f640 1 -- 172.21.15.160:0/3294071914 shutdown_connections 2024-09-16T08:40:28.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:28.261+0000 7fd78213f640 1 -- 172.21.15.160:0/3294071914 wait complete. 2024-09-16T08:40:28.854 INFO:teuthology.orchestra.run.smithi160.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-16T08:40:28.854 INFO:teuthology.orchestra.run.smithi160.stdout:Adding key to root@localhost authorized_keys... 2024-09-16T08:40:28.859 INFO:teuthology.orchestra.run.smithi160.stdout:Adding host smithi160... 2024-09-16T08:40:29.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:29 smithi160 ceph-mon[30137]: from='client.14136 v1:172.21.15.160:0/3294071914' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:40:30.275 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:30 smithi160 ceph-mon[30137]: from='client.14138 v1:172.21.15.160:0/959096118' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi160", "addr": "172.21.15.160", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:40:31.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:31 smithi160 ceph-mon[30137]: Deploying cephadm binary to smithi160 2024-09-16T08:40:33.187 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout Added host 'smithi160' with addr '172.21.15.160' 2024-09-16T08:40:33.187 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.324+0000 7f5fc4ece640 1 Processor -- start 2024-09-16T08:40:33.187 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.324+0000 7f5fc4ece640 1 -- start start 2024-09-16T08:40:33.187 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.324+0000 7f5fc4ece640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f5fc010ab90 con 0x7f5fc0106710 2024-09-16T08:40:33.187 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.324+0000 7f5fbe59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f5fc0106710 0x7f5fc0106b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:52856/0 (socket says 172.21.15.160:52856) 2024-09-16T08:40:33.187 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.324+0000 7f5fbe59c640 1 -- 172.21.15.160:0/1951248269 learned_addr learned my addr 172.21.15.160:0/1951248269 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:33.187 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.325+0000 7f5fbd59a640 1 -- 172.21.15.160:0/1951248269 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3998370957 0 0) 0x7f5fc010ab90 con 0x7f5fc0106710 2024-09-16T08:40:33.188 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.325+0000 7f5fbd59a640 1 -- 172.21.15.160:0/1951248269 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fa8003520 con 0x7f5fc0106710 2024-09-16T08:40:33.188 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.325+0000 7f5fbd59a640 1 -- 172.21.15.160:0/1951248269 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1998880255 0 0) 0x7f5fa8003520 con 0x7f5fc0106710 2024-09-16T08:40:33.188 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.325+0000 7f5fbd59a640 1 -- 172.21.15.160:0/1951248269 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc010bd70 con 0x7f5fc0106710 2024-09-16T08:40:33.188 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.325+0000 7f5fbd59a640 1 -- 172.21.15.160:0/1951248269 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f5fb0001600 con 0x7f5fc0106710 2024-09-16T08:40:33.188 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.325+0000 7f5fbd59a640 1 -- 172.21.15.160:0/1951248269 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f5fb0001cf0 con 0x7f5fc0106710 2024-09-16T08:40:33.188 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.325+0000 7f5fbd59a640 1 -- 172.21.15.160:0/1951248269 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f5fb00058d0 con 0x7f5fc0106710 2024-09-16T08:40:33.188 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.325+0000 7f5fc4ece640 1 -- 172.21.15.160:0/1951248269 >> v1:172.21.15.160:6789/0 conn(0x7f5fc0106710 legacy=0x7f5fc0106b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:33.188 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.325+0000 7f5fc4ece640 1 -- 172.21.15.160:0/1951248269 shutdown_connections 2024-09-16T08:40:33.188 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.325+0000 7f5fc4ece640 1 -- 172.21.15.160:0/1951248269 >> 172.21.15.160:0/1951248269 conn(0x7f5fc0102000 msgr2=0x7f5fc0104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:33.188 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.326+0000 7f5fc4ece640 1 -- 172.21.15.160:0/1951248269 shutdown_connections 2024-09-16T08:40:33.188 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.326+0000 7f5fc4ece640 1 -- 172.21.15.160:0/1951248269 wait complete. 2024-09-16T08:40:33.188 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.326+0000 7f5fc4ece640 1 Processor -- start 2024-09-16T08:40:33.188 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.326+0000 7f5fc4ece640 1 -- start start 2024-09-16T08:40:33.189 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.326+0000 7f5fc4ece640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f5fc019aa30 con 0x7f5fc0106710 2024-09-16T08:40:33.189 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.327+0000 7f5fbe59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f5fc0106710 0x7f5fc019a2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:52860/0 (socket says 172.21.15.160:52860) 2024-09-16T08:40:33.189 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.327+0000 7f5fbe59c640 1 -- 172.21.15.160:0/959096118 learned_addr learned my addr 172.21.15.160:0/959096118 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:33.189 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.327+0000 7f5faf7fe640 1 -- 172.21.15.160:0/959096118 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1849315779 0 0) 0x7f5fc019aa30 con 0x7f5fc0106710 2024-09-16T08:40:33.189 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.327+0000 7f5faf7fe640 1 -- 172.21.15.160:0/959096118 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fa8005570 con 0x7f5fc0106710 2024-09-16T08:40:33.189 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.327+0000 7f5faf7fe640 1 -- 172.21.15.160:0/959096118 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3390924375 0 0) 0x7f5fa8005570 con 0x7f5fc0106710 2024-09-16T08:40:33.189 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.327+0000 7f5faf7fe640 1 -- 172.21.15.160:0/959096118 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5fc019aa30 con 0x7f5fc0106710 2024-09-16T08:40:33.189 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.327+0000 7f5faf7fe640 1 -- 172.21.15.160:0/959096118 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f5fb00047e0 con 0x7f5fc0106710 2024-09-16T08:40:33.189 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.327+0000 7f5faf7fe640 1 -- 172.21.15.160:0/959096118 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3527198284 0 0) 0x7f5fc019aa30 con 0x7f5fc0106710 2024-09-16T08:40:33.189 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.327+0000 7f5faf7fe640 1 -- 172.21.15.160:0/959096118 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc019ac20 con 0x7f5fc0106710 2024-09-16T08:40:33.189 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.327+0000 7f5fc4ece640 1 -- 172.21.15.160:0/959096118 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc019e500 con 0x7f5fc0106710 2024-09-16T08:40:33.189 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.327+0000 7f5fc4ece640 1 -- 172.21.15.160:0/959096118 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f5fc019e920 con 0x7f5fc0106710 2024-09-16T08:40:33.190 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.327+0000 7f5faf7fe640 1 -- 172.21.15.160:0/959096118 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7f5fb0001bb0 con 0x7f5fc0106710 2024-09-16T08:40:33.190 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.327+0000 7f5faf7fe640 1 -- 172.21.15.160:0/959096118 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f5fb0005f80 con 0x7f5fc0106710 2024-09-16T08:40:33.190 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.328+0000 7f5faf7fe640 1 -- 172.21.15.160:0/959096118 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 8) ==== 51730+0+0 (unknown 4165010166 0 0) 0x7f5fb0012bc0 con 0x7f5fc0106710 2024-09-16T08:40:33.190 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.328+0000 7f5fc4ece640 1 -- 172.21.15.160:0/959096118 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fa00050b0 con 0x7f5fc0106710 2024-09-16T08:40:33.190 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.328+0000 7f5faf7fe640 1 -- 172.21.15.160:0/959096118 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 760617087 0 0) 0x7f5fb004f790 con 0x7f5fc0106710 2024-09-16T08:40:33.190 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.333+0000 7f5faf7fe640 1 -- 172.21.15.160:0/959096118 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f5fb0005730 con 0x7f5fc0106710 2024-09-16T08:40:33.190 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:29.514+0000 7f5fc4ece640 1 -- 172.21.15.160:0/959096118 --> v1:172.21.15.160:6800/3425583652 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi160", "addr": "172.21.15.160", "target": ["mon-mgr", ""]}) -- 0x7f5fa0000bc0 con 0x7f5fa8040870 2024-09-16T08:40:33.190 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:32.588+0000 7f5faf7fe640 1 -- 172.21.15.160:0/959096118 <== mgr.14116 v1:172.21.15.160:6800/3425583652 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (unknown 0 0 1016171551) 0x7f5fa0000bc0 con 0x7f5fa8040870 2024-09-16T08:40:33.190 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:32.591+0000 7f5fc4ece640 1 -- 172.21.15.160:0/959096118 >> v1:172.21.15.160:6800/3425583652 conn(0x7f5fa8040870 legacy=0x7f5fa8042d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:33.190 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:32.591+0000 7f5fc4ece640 1 -- 172.21.15.160:0/959096118 >> v1:172.21.15.160:6789/0 conn(0x7f5fc0106710 legacy=0x7f5fc019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:33.190 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:32.592+0000 7f5fc4ece640 1 -- 172.21.15.160:0/959096118 shutdown_connections 2024-09-16T08:40:33.190 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:32.592+0000 7f5fc4ece640 1 -- 172.21.15.160:0/959096118 >> 172.21.15.160:0/959096118 conn(0x7f5fc0102000 msgr2=0x7f5fc01043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:33.190 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:32.592+0000 7f5fc4ece640 1 -- 172.21.15.160:0/959096118 shutdown_connections 2024-09-16T08:40:33.191 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:32.592+0000 7f5fc4ece640 1 -- 172.21.15.160:0/959096118 wait complete. 2024-09-16T08:40:33.191 INFO:teuthology.orchestra.run.smithi160.stdout:Deploying unmanaged mon service... 2024-09-16T08:40:33.831 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:33 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' 2024-09-16T08:40:33.831 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:33 smithi160 ceph-mon[30137]: Added host smithi160 2024-09-16T08:40:33.831 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:33 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:40:34.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-16T08:40:34.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.632+0000 7fd3a2a60640 1 Processor -- start 2024-09-16T08:40:34.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.632+0000 7fd3a2a60640 1 -- start start 2024-09-16T08:40:34.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.632+0000 7fd3a2a60640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd39c10ac10 con 0x7fd39c106790 2024-09-16T08:40:34.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.633+0000 7fd39bfff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd39c106790 0x7fd39c106b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:52864/0 (socket says 172.21.15.160:52864) 2024-09-16T08:40:34.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.633+0000 7fd39bfff640 1 -- 172.21.15.160:0/4197654363 learned_addr learned my addr 172.21.15.160:0/4197654363 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:34.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.633+0000 7fd39affd640 1 -- 172.21.15.160:0/4197654363 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1321388759 0 0) 0x7fd39c10ac10 con 0x7fd39c106790 2024-09-16T08:40:34.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.633+0000 7fd39affd640 1 -- 172.21.15.160:0/4197654363 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd384003520 con 0x7fd39c106790 2024-09-16T08:40:34.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.633+0000 7fd39affd640 1 -- 172.21.15.160:0/4197654363 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1587863399 0 0) 0x7fd384003520 con 0x7fd39c106790 2024-09-16T08:40:34.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.633+0000 7fd39affd640 1 -- 172.21.15.160:0/4197654363 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39c10bdf0 con 0x7fd39c106790 2024-09-16T08:40:34.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.633+0000 7fd39affd640 1 -- 172.21.15.160:0/4197654363 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fd38c001600 con 0x7fd39c106790 2024-09-16T08:40:34.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.633+0000 7fd39affd640 1 -- 172.21.15.160:0/4197654363 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7fd38c001cf0 con 0x7fd39c106790 2024-09-16T08:40:34.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.633+0000 7fd39affd640 1 -- 172.21.15.160:0/4197654363 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fd38c0058d0 con 0x7fd39c106790 2024-09-16T08:40:34.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.634+0000 7fd3a2a60640 1 -- 172.21.15.160:0/4197654363 >> v1:172.21.15.160:6789/0 conn(0x7fd39c106790 legacy=0x7fd39c106b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:34.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.634+0000 7fd3a2a60640 1 -- 172.21.15.160:0/4197654363 shutdown_connections 2024-09-16T08:40:34.563 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.634+0000 7fd3a2a60640 1 -- 172.21.15.160:0/4197654363 >> 172.21.15.160:0/4197654363 conn(0x7fd39c102080 msgr2=0x7fd39c1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:34.563 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.634+0000 7fd3a2a60640 1 -- 172.21.15.160:0/4197654363 shutdown_connections 2024-09-16T08:40:34.563 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.634+0000 7fd3a2a60640 1 -- 172.21.15.160:0/4197654363 wait complete. 2024-09-16T08:40:34.563 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.634+0000 7fd3a2a60640 1 Processor -- start 2024-09-16T08:40:34.563 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.634+0000 7fd3a2a60640 1 -- start start 2024-09-16T08:40:34.563 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.635+0000 7fd3a2a60640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd39c19aa60 con 0x7fd39c106790 2024-09-16T08:40:34.563 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.635+0000 7fd39bfff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd39c106790 0x7fd39c19a300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:52868/0 (socket says 172.21.15.160:52868) 2024-09-16T08:40:34.563 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.635+0000 7fd39bfff640 1 -- 172.21.15.160:0/2948837340 learned_addr learned my addr 172.21.15.160:0/2948837340 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:34.563 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.635+0000 7fd3997fa640 1 -- 172.21.15.160:0/2948837340 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2514528156 0 0) 0x7fd39c19aa60 con 0x7fd39c106790 2024-09-16T08:40:34.563 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.635+0000 7fd3997fa640 1 -- 172.21.15.160:0/2948837340 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd384005570 con 0x7fd39c106790 2024-09-16T08:40:34.563 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.635+0000 7fd3997fa640 1 -- 172.21.15.160:0/2948837340 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2946669900 0 0) 0x7fd384005570 con 0x7fd39c106790 2024-09-16T08:40:34.564 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.635+0000 7fd3997fa640 1 -- 172.21.15.160:0/2948837340 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd39c19aa60 con 0x7fd39c106790 2024-09-16T08:40:34.564 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.635+0000 7fd3997fa640 1 -- 172.21.15.160:0/2948837340 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fd38c001080 con 0x7fd39c106790 2024-09-16T08:40:34.564 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.635+0000 7fd3997fa640 1 -- 172.21.15.160:0/2948837340 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1187005629 0 0) 0x7fd39c19aa60 con 0x7fd39c106790 2024-09-16T08:40:34.564 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.635+0000 7fd3997fa640 1 -- 172.21.15.160:0/2948837340 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39c19ac50 con 0x7fd39c106790 2024-09-16T08:40:34.564 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.635+0000 7fd3a2a60640 1 -- 172.21.15.160:0/2948837340 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd39c19e530 con 0x7fd39c106790 2024-09-16T08:40:34.564 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.636+0000 7fd3a2a60640 1 -- 172.21.15.160:0/2948837340 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd39c19e950 con 0x7fd39c106790 2024-09-16T08:40:34.564 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.636+0000 7fd3997fa640 1 -- 172.21.15.160:0/2948837340 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7fd38c001820 con 0x7fd39c106790 2024-09-16T08:40:34.564 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.636+0000 7fd3997fa640 1 -- 172.21.15.160:0/2948837340 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fd38c006030 con 0x7fd39c106790 2024-09-16T08:40:34.564 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.637+0000 7fd3997fa640 1 -- 172.21.15.160:0/2948837340 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 8) ==== 51730+0+0 (unknown 4165010166 0 0) 0x7fd38c012c70 con 0x7fd39c106790 2024-09-16T08:40:34.564 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.637+0000 7fd3a2a60640 1 -- 172.21.15.160:0/2948837340 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3880050b0 con 0x7fd39c106790 2024-09-16T08:40:34.564 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.637+0000 7fd3997fa640 1 -- 172.21.15.160:0/2948837340 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 760617087 0 0) 0x7fd38c04f8a0 con 0x7fd39c106790 2024-09-16T08:40:34.564 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.641+0000 7fd3997fa640 1 -- 172.21.15.160:0/2948837340 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fd38c018ef0 con 0x7fd39c106790 2024-09-16T08:40:34.564 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.830+0000 7fd3a2a60640 1 -- 172.21.15.160:0/2948837340 --> v1:172.21.15.160:6800/3425583652 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fd388000bc0 con 0x7fd384040820 2024-09-16T08:40:34.565 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.835+0000 7fd3997fa640 1 -- 172.21.15.160:0/2948837340 <== mgr.14116 v1:172.21.15.160:6800/3425583652 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (unknown 0 0 3265049985) 0x7fd388000bc0 con 0x7fd384040820 2024-09-16T08:40:34.565 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.837+0000 7fd3a2a60640 1 -- 172.21.15.160:0/2948837340 >> v1:172.21.15.160:6800/3425583652 conn(0x7fd384040820 legacy=0x7fd384042ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:34.565 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.837+0000 7fd3a2a60640 1 -- 172.21.15.160:0/2948837340 >> v1:172.21.15.160:6789/0 conn(0x7fd39c106790 legacy=0x7fd39c19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:34.565 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.837+0000 7fd3a2a60640 1 -- 172.21.15.160:0/2948837340 shutdown_connections 2024-09-16T08:40:34.565 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.837+0000 7fd3a2a60640 1 -- 172.21.15.160:0/2948837340 >> 172.21.15.160:0/2948837340 conn(0x7fd39c102080 msgr2=0x7fd39c104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:34.565 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.837+0000 7fd3a2a60640 1 -- 172.21.15.160:0/2948837340 shutdown_connections 2024-09-16T08:40:34.565 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:33.837+0000 7fd3a2a60640 1 -- 172.21.15.160:0/2948837340 wait complete. 2024-09-16T08:40:34.565 INFO:teuthology.orchestra.run.smithi160.stdout:Deploying unmanaged mgr service... 2024-09-16T08:40:35.159 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:34 smithi160 ceph-mon[30137]: from='client.14140 v1:172.21.15.160:0/2948837340' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:40:35.159 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:34 smithi160 ceph-mon[30137]: Saving service mon spec with placement count:5 2024-09-16T08:40:35.159 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:34 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' 2024-09-16T08:40:36.565 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-16T08:40:36.566 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.239+0000 7fa892392640 1 Processor -- start 2024-09-16T08:40:36.566 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.239+0000 7fa892392640 1 -- start start 2024-09-16T08:40:36.566 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.239+0000 7fa892392640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa88c109ac0 con 0x7fa88c105640 2024-09-16T08:40:36.566 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.239+0000 7fa88bfff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fa88c105640 0x7fa88c105a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:52884/0 (socket says 172.21.15.160:52884) 2024-09-16T08:40:36.566 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.239+0000 7fa88bfff640 1 -- 172.21.15.160:0/1325939399 learned_addr learned my addr 172.21.15.160:0/1325939399 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:36.566 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.239+0000 7fa88affd640 1 -- 172.21.15.160:0/1325939399 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2220860011 0 0) 0x7fa88c109ac0 con 0x7fa88c105640 2024-09-16T08:40:36.566 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.239+0000 7fa88affd640 1 -- 172.21.15.160:0/1325939399 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa874003520 con 0x7fa88c105640 2024-09-16T08:40:36.567 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.240+0000 7fa88affd640 1 -- 172.21.15.160:0/1325939399 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2409375538 0 0) 0x7fa874003520 con 0x7fa88c105640 2024-09-16T08:40:36.567 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.240+0000 7fa88affd640 1 -- 172.21.15.160:0/1325939399 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88c10aca0 con 0x7fa88c105640 2024-09-16T08:40:36.567 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.240+0000 7fa88affd640 1 -- 172.21.15.160:0/1325939399 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fa87c001600 con 0x7fa88c105640 2024-09-16T08:40:36.567 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.240+0000 7fa88affd640 1 -- 172.21.15.160:0/1325939399 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7fa87c001cf0 con 0x7fa88c105640 2024-09-16T08:40:36.567 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.240+0000 7fa88affd640 1 -- 172.21.15.160:0/1325939399 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fa87c0058d0 con 0x7fa88c105640 2024-09-16T08:40:36.567 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.240+0000 7fa892392640 1 -- 172.21.15.160:0/1325939399 >> v1:172.21.15.160:6789/0 conn(0x7fa88c105640 legacy=0x7fa88c105a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:36.567 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.240+0000 7fa892392640 1 -- 172.21.15.160:0/1325939399 shutdown_connections 2024-09-16T08:40:36.567 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.240+0000 7fa892392640 1 -- 172.21.15.160:0/1325939399 >> 172.21.15.160:0/1325939399 conn(0x7fa88c100f30 msgr2=0x7fa88c103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:36.567 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.240+0000 7fa892392640 1 -- 172.21.15.160:0/1325939399 shutdown_connections 2024-09-16T08:40:36.567 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.240+0000 7fa892392640 1 -- 172.21.15.160:0/1325939399 wait complete. 2024-09-16T08:40:36.567 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.241+0000 7fa892392640 1 Processor -- start 2024-09-16T08:40:36.567 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.241+0000 7fa892392640 1 -- start start 2024-09-16T08:40:36.568 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.241+0000 7fa892392640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa88c19e020 con 0x7fa88c105640 2024-09-16T08:40:36.568 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.241+0000 7fa88bfff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fa88c105640 0x7fa88c07a050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:52888/0 (socket says 172.21.15.160:52888) 2024-09-16T08:40:36.568 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.241+0000 7fa88bfff640 1 -- 172.21.15.160:0/1107344577 learned_addr learned my addr 172.21.15.160:0/1107344577 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:36.568 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.241+0000 7fa8897fa640 1 -- 172.21.15.160:0/1107344577 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4143680091 0 0) 0x7fa88c19e020 con 0x7fa88c105640 2024-09-16T08:40:36.568 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.241+0000 7fa8897fa640 1 -- 172.21.15.160:0/1107344577 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa874005570 con 0x7fa88c105640 2024-09-16T08:40:36.568 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.242+0000 7fa8897fa640 1 -- 172.21.15.160:0/1107344577 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1130654914 0 0) 0x7fa874005570 con 0x7fa88c105640 2024-09-16T08:40:36.568 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.242+0000 7fa8897fa640 1 -- 172.21.15.160:0/1107344577 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa88c19e020 con 0x7fa88c105640 2024-09-16T08:40:36.568 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.242+0000 7fa8897fa640 1 -- 172.21.15.160:0/1107344577 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fa87c001080 con 0x7fa88c105640 2024-09-16T08:40:36.568 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.242+0000 7fa8897fa640 1 -- 172.21.15.160:0/1107344577 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1591447825 0 0) 0x7fa88c19e020 con 0x7fa88c105640 2024-09-16T08:40:36.569 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.242+0000 7fa8897fa640 1 -- 172.21.15.160:0/1107344577 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88c19f200 con 0x7fa88c105640 2024-09-16T08:40:36.569 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.242+0000 7fa892392640 1 -- 172.21.15.160:0/1107344577 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa88c19e1f0 con 0x7fa88c105640 2024-09-16T08:40:36.569 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.242+0000 7fa892392640 1 -- 172.21.15.160:0/1107344577 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa88c19e780 con 0x7fa88c105640 2024-09-16T08:40:36.569 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.242+0000 7fa8897fa640 1 -- 172.21.15.160:0/1107344577 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 998+0+0 (unknown 1209452942 0 0) 0x7fa87c001820 con 0x7fa88c105640 2024-09-16T08:40:36.569 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.242+0000 7fa8897fa640 1 -- 172.21.15.160:0/1107344577 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fa87c006030 con 0x7fa88c105640 2024-09-16T08:40:36.569 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.243+0000 7fa8897fa640 1 -- 172.21.15.160:0/1107344577 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 8) ==== 51730+0+0 (unknown 4165010166 0 0) 0x7fa87c012c70 con 0x7fa88c105640 2024-09-16T08:40:36.569 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.243+0000 7fa892392640 1 -- 172.21.15.160:0/1107344577 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8780050b0 con 0x7fa88c105640 2024-09-16T08:40:36.570 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.243+0000 7fa8897fa640 1 -- 172.21.15.160:0/1107344577 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 760617087 0 0) 0x7fa87c04f7e0 con 0x7fa88c105640 2024-09-16T08:40:36.570 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.248+0000 7fa8897fa640 1 -- 172.21.15.160:0/1107344577 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fa87c018ef0 con 0x7fa88c105640 2024-09-16T08:40:36.570 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.439+0000 7fa892392640 1 -- 172.21.15.160:0/1107344577 --> v1:172.21.15.160:6800/3425583652 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa878000bc0 con 0x7fa874040820 2024-09-16T08:40:36.570 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.444+0000 7fa8897fa640 1 -- 172.21.15.160:0/1107344577 <== mgr.14116 v1:172.21.15.160:6800/3425583652 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (unknown 0 0 325935098) 0x7fa878000bc0 con 0x7fa874040820 2024-09-16T08:40:36.570 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.446+0000 7fa892392640 1 -- 172.21.15.160:0/1107344577 >> v1:172.21.15.160:6800/3425583652 conn(0x7fa874040820 legacy=0x7fa874042ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:36.570 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.446+0000 7fa892392640 1 -- 172.21.15.160:0/1107344577 >> v1:172.21.15.160:6789/0 conn(0x7fa88c105640 legacy=0x7fa88c07a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:36.570 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.446+0000 7fa892392640 1 -- 172.21.15.160:0/1107344577 shutdown_connections 2024-09-16T08:40:36.570 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.446+0000 7fa892392640 1 -- 172.21.15.160:0/1107344577 >> 172.21.15.160:0/1107344577 conn(0x7fa88c100f30 msgr2=0x7fa88c103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:36.571 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.446+0000 7fa892392640 1 -- 172.21.15.160:0/1107344577 shutdown_connections 2024-09-16T08:40:36.571 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:35.447+0000 7fa892392640 1 -- 172.21.15.160:0/1107344577 wait complete. 2024-09-16T08:40:36.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:36 smithi160 ceph-mon[30137]: from='client.14142 v1:172.21.15.160:0/1107344577' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:40:36.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:36 smithi160 ceph-mon[30137]: Saving service mgr spec with placement count:2 2024-09-16T08:40:36.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:36 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' 2024-09-16T08:40:36.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:36 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' 2024-09-16T08:40:37.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:37 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' 2024-09-16T08:40:37.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3033510777' entity='client.admin' 2024-09-16T08:40:38.022 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.051+0000 7faa72297640 1 Processor -- start 2024-09-16T08:40:38.023 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.051+0000 7faa72297640 1 -- start start 2024-09-16T08:40:38.023 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.051+0000 7faa72297640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7faa6c10ab90 con 0x7faa6c106710 2024-09-16T08:40:38.023 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.051+0000 7faa6bfff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7faa6c106710 0x7faa6c106b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35262/0 (socket says 172.21.15.160:35262) 2024-09-16T08:40:38.023 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.051+0000 7faa6bfff640 1 -- 172.21.15.160:0/837267516 learned_addr learned my addr 172.21.15.160:0/837267516 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:38.023 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.051+0000 7faa6affd640 1 -- 172.21.15.160:0/837267516 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2275825713 0 0) 0x7faa6c10ab90 con 0x7faa6c106710 2024-09-16T08:40:38.023 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.052+0000 7faa6affd640 1 -- 172.21.15.160:0/837267516 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa54003520 con 0x7faa6c106710 2024-09-16T08:40:38.023 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.052+0000 7faa6affd640 1 -- 172.21.15.160:0/837267516 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3077766860 0 0) 0x7faa54003520 con 0x7faa6c106710 2024-09-16T08:40:38.023 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.052+0000 7faa6affd640 1 -- 172.21.15.160:0/837267516 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c10bd70 con 0x7faa6c106710 2024-09-16T08:40:38.024 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.052+0000 7faa6affd640 1 -- 172.21.15.160:0/837267516 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7faa5c001600 con 0x7faa6c106710 2024-09-16T08:40:38.024 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.052+0000 7faa6affd640 1 -- 172.21.15.160:0/837267516 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7faa5c001c10 con 0x7faa6c106710 2024-09-16T08:40:38.024 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.052+0000 7faa6affd640 1 -- 172.21.15.160:0/837267516 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7faa5c0058d0 con 0x7faa6c106710 2024-09-16T08:40:38.024 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.052+0000 7faa72297640 1 -- 172.21.15.160:0/837267516 >> v1:172.21.15.160:6789/0 conn(0x7faa6c106710 legacy=0x7faa6c106b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:38.024 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.052+0000 7faa72297640 1 -- 172.21.15.160:0/837267516 shutdown_connections 2024-09-16T08:40:38.024 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.052+0000 7faa72297640 1 -- 172.21.15.160:0/837267516 >> 172.21.15.160:0/837267516 conn(0x7faa6c102000 msgr2=0x7faa6c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:38.024 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.052+0000 7faa72297640 1 -- 172.21.15.160:0/837267516 shutdown_connections 2024-09-16T08:40:38.024 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.052+0000 7faa72297640 1 -- 172.21.15.160:0/837267516 wait complete. 2024-09-16T08:40:38.024 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.053+0000 7faa72297640 1 Processor -- start 2024-09-16T08:40:38.024 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.053+0000 7faa72297640 1 -- start start 2024-09-16T08:40:38.024 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.053+0000 7faa72297640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7faa6c19ab40 con 0x7faa6c106710 2024-09-16T08:40:38.025 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.053+0000 7faa6bfff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7faa6c106710 0x7faa6c19a3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35268/0 (socket says 172.21.15.160:35268) 2024-09-16T08:40:38.025 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.053+0000 7faa6bfff640 1 -- 172.21.15.160:0/3033510777 learned_addr learned my addr 172.21.15.160:0/3033510777 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:38.025 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.054+0000 7faa697fa640 1 -- 172.21.15.160:0/3033510777 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1985649291 0 0) 0x7faa6c19ab40 con 0x7faa6c106710 2024-09-16T08:40:38.025 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.054+0000 7faa697fa640 1 -- 172.21.15.160:0/3033510777 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa540055c0 con 0x7faa6c106710 2024-09-16T08:40:38.025 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.054+0000 7faa697fa640 1 -- 172.21.15.160:0/3033510777 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3796960504 0 0) 0x7faa540055c0 con 0x7faa6c106710 2024-09-16T08:40:38.025 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.054+0000 7faa697fa640 1 -- 172.21.15.160:0/3033510777 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7faa6c19ab40 con 0x7faa6c106710 2024-09-16T08:40:38.025 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.054+0000 7faa697fa640 1 -- 172.21.15.160:0/3033510777 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7faa5c001080 con 0x7faa6c106710 2024-09-16T08:40:38.025 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.054+0000 7faa697fa640 1 -- 172.21.15.160:0/3033510777 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3871514714 0 0) 0x7faa6c19ab40 con 0x7faa6c106710 2024-09-16T08:40:38.025 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.054+0000 7faa697fa640 1 -- 172.21.15.160:0/3033510777 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c19ad30 con 0x7faa6c106710 2024-09-16T08:40:38.025 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.054+0000 7faa72297640 1 -- 172.21.15.160:0/3033510777 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c19e610 con 0x7faa6c106710 2024-09-16T08:40:38.025 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.054+0000 7faa72297640 1 -- 172.21.15.160:0/3033510777 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7faa6c19ea30 con 0x7faa6c106710 2024-09-16T08:40:38.025 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.054+0000 7faa697fa640 1 -- 172.21.15.160:0/3033510777 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7faa5c004bf0 con 0x7faa6c106710 2024-09-16T08:40:38.025 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.054+0000 7faa697fa640 1 -- 172.21.15.160:0/3033510777 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7faa5c005f10 con 0x7faa6c106710 2024-09-16T08:40:38.026 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.055+0000 7faa697fa640 1 -- 172.21.15.160:0/3033510777 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 8) ==== 51730+0+0 (unknown 4165010166 0 0) 0x7faa5c012af0 con 0x7faa6c106710 2024-09-16T08:40:38.026 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.055+0000 7faa72297640 1 -- 172.21.15.160:0/3033510777 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa580050b0 con 0x7faa6c106710 2024-09-16T08:40:38.026 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.055+0000 7faa697fa640 1 -- 172.21.15.160:0/3033510777 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 760617087 0 0) 0x7faa5c04f690 con 0x7faa6c106710 2024-09-16T08:40:38.026 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.060+0000 7faa697fa640 1 -- 172.21.15.160:0/3033510777 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7faa5c018d20 con 0x7faa6c106710 2024-09-16T08:40:38.026 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.251+0000 7faa72297640 1 -- 172.21.15.160:0/3033510777 --> v1:172.21.15.160:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7faa580053a0 con 0x7faa6c106710 2024-09-16T08:40:38.026 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.254+0000 7faa697fa640 1 -- 172.21.15.160:0/3033510777 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (unknown 2684808024 0 0) 0x7faa5c018620 con 0x7faa6c106710 2024-09-16T08:40:38.026 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.256+0000 7faa72297640 1 -- 172.21.15.160:0/3033510777 >> v1:172.21.15.160:6800/3425583652 conn(0x7faa540408b0 legacy=0x7faa54042d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:38.026 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.256+0000 7faa72297640 1 -- 172.21.15.160:0/3033510777 >> v1:172.21.15.160:6789/0 conn(0x7faa6c106710 legacy=0x7faa6c19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:38.026 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.256+0000 7faa72297640 1 -- 172.21.15.160:0/3033510777 shutdown_connections 2024-09-16T08:40:38.026 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.256+0000 7faa72297640 1 -- 172.21.15.160:0/3033510777 >> 172.21.15.160:0/3033510777 conn(0x7faa6c102000 msgr2=0x7faa6c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:38.026 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.257+0000 7faa72297640 1 -- 172.21.15.160:0/3033510777 shutdown_connections 2024-09-16T08:40:38.026 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:37.257+0000 7faa72297640 1 -- 172.21.15.160:0/3033510777 wait complete. 2024-09-16T08:40:39.373 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.467+0000 7f5052eba640 1 Processor -- start 2024-09-16T08:40:39.373 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.467+0000 7f5052eba640 1 -- start start 2024-09-16T08:40:39.373 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.467+0000 7f5052eba640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f504c10ab90 con 0x7f504c106710 2024-09-16T08:40:39.373 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.468+0000 7f5050c56640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f504c106710 0x7f504c106b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35276/0 (socket says 172.21.15.160:35276) 2024-09-16T08:40:39.373 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.468+0000 7f5050c56640 1 -- 172.21.15.160:0/1713759745 learned_addr learned my addr 172.21.15.160:0/1713759745 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:39.373 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.468+0000 7f50437fe640 1 -- 172.21.15.160:0/1713759745 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1766613328 0 0) 0x7f504c10ab90 con 0x7f504c106710 2024-09-16T08:40:39.373 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.468+0000 7f50437fe640 1 -- 172.21.15.160:0/1713759745 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5034003520 con 0x7f504c106710 2024-09-16T08:40:39.373 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.468+0000 7f50437fe640 1 -- 172.21.15.160:0/1713759745 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3516702934 0 0) 0x7f5034003520 con 0x7f504c106710 2024-09-16T08:40:39.373 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.468+0000 7f50437fe640 1 -- 172.21.15.160:0/1713759745 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c10bd70 con 0x7f504c106710 2024-09-16T08:40:39.373 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.468+0000 7f50437fe640 1 -- 172.21.15.160:0/1713759745 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f5048001600 con 0x7f504c106710 2024-09-16T08:40:39.373 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.468+0000 7f50437fe640 1 -- 172.21.15.160:0/1713759745 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f5048001c10 con 0x7f504c106710 2024-09-16T08:40:39.374 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.468+0000 7f50437fe640 1 -- 172.21.15.160:0/1713759745 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f50480058d0 con 0x7f504c106710 2024-09-16T08:40:39.374 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.469+0000 7f5052eba640 1 -- 172.21.15.160:0/1713759745 >> v1:172.21.15.160:6789/0 conn(0x7f504c106710 legacy=0x7f504c106b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:39.374 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.469+0000 7f5052eba640 1 -- 172.21.15.160:0/1713759745 shutdown_connections 2024-09-16T08:40:39.374 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.469+0000 7f5052eba640 1 -- 172.21.15.160:0/1713759745 >> 172.21.15.160:0/1713759745 conn(0x7f504c102000 msgr2=0x7f504c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:39.374 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.469+0000 7f5052eba640 1 -- 172.21.15.160:0/1713759745 shutdown_connections 2024-09-16T08:40:39.374 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.469+0000 7f5052eba640 1 -- 172.21.15.160:0/1713759745 wait complete. 2024-09-16T08:40:39.374 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.469+0000 7f5052eba640 1 Processor -- start 2024-09-16T08:40:39.374 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.469+0000 7f5052eba640 1 -- start start 2024-09-16T08:40:39.374 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.469+0000 7f5052eba640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f504c19ab10 con 0x7f504c106710 2024-09-16T08:40:39.374 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.469+0000 7f5050c56640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f504c106710 0x7f504c19a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35292/0 (socket says 172.21.15.160:35292) 2024-09-16T08:40:39.374 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.470+0000 7f5050c56640 1 -- 172.21.15.160:0/1860363171 learned_addr learned my addr 172.21.15.160:0/1860363171 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:39.374 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.470+0000 7f5041ffb640 1 -- 172.21.15.160:0/1860363171 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 260414386 0 0) 0x7f504c19ab10 con 0x7f504c106710 2024-09-16T08:40:39.374 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.470+0000 7f5041ffb640 1 -- 172.21.15.160:0/1860363171 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50340055c0 con 0x7f504c106710 2024-09-16T08:40:39.375 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.470+0000 7f5041ffb640 1 -- 172.21.15.160:0/1860363171 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 901323331 0 0) 0x7f50340055c0 con 0x7f504c106710 2024-09-16T08:40:39.375 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.470+0000 7f5041ffb640 1 -- 172.21.15.160:0/1860363171 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f504c19ab10 con 0x7f504c106710 2024-09-16T08:40:39.375 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.470+0000 7f5041ffb640 1 -- 172.21.15.160:0/1860363171 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f5048001080 con 0x7f504c106710 2024-09-16T08:40:39.375 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.470+0000 7f5041ffb640 1 -- 172.21.15.160:0/1860363171 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 387090267 0 0) 0x7f504c19ab10 con 0x7f504c106710 2024-09-16T08:40:39.375 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.470+0000 7f5041ffb640 1 -- 172.21.15.160:0/1860363171 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c19ad00 con 0x7f504c106710 2024-09-16T08:40:39.375 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.470+0000 7f5052eba640 1 -- 172.21.15.160:0/1860363171 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f504c19e5e0 con 0x7f504c106710 2024-09-16T08:40:39.375 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.470+0000 7f5041ffb640 1 -- 172.21.15.160:0/1860363171 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f5048005520 con 0x7f504c106710 2024-09-16T08:40:39.375 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.470+0000 7f5041ffb640 1 -- 172.21.15.160:0/1860363171 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f5048006230 con 0x7f504c106710 2024-09-16T08:40:39.376 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.470+0000 7f5052eba640 1 -- 172.21.15.160:0/1860363171 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f504c19ea00 con 0x7f504c106710 2024-09-16T08:40:39.376 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.471+0000 7f5041ffb640 1 -- 172.21.15.160:0/1860363171 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 8) ==== 51730+0+0 (unknown 4165010166 0 0) 0x7f5048012e90 con 0x7f504c106710 2024-09-16T08:40:39.376 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.471+0000 7f5052eba640 1 -- 172.21.15.160:0/1860363171 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f504c193710 con 0x7f504c106710 2024-09-16T08:40:39.376 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.471+0000 7f5041ffb640 1 -- 172.21.15.160:0/1860363171 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 760617087 0 0) 0x7f504804f8e0 con 0x7f504c106710 2024-09-16T08:40:39.376 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.476+0000 7f5041ffb640 1 -- 172.21.15.160:0/1860363171 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f5048018d90 con 0x7f504c106710 2024-09-16T08:40:39.376 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.653+0000 7f5052eba640 1 -- 172.21.15.160:0/1860363171 --> v1:172.21.15.160:6789/0 -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f504c10b910 con 0x7f504c106710 2024-09-16T08:40:39.376 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.655+0000 7f5041ffb640 1 -- 172.21.15.160:0/1860363171 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (unknown 1496729260 0 0) 0x7f5048018690 con 0x7f504c106710 2024-09-16T08:40:39.376 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.658+0000 7f5052eba640 1 -- 172.21.15.160:0/1860363171 >> v1:172.21.15.160:6800/3425583652 conn(0x7f5034040900 legacy=0x7f5034042dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:39.377 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.658+0000 7f5052eba640 1 -- 172.21.15.160:0/1860363171 >> v1:172.21.15.160:6789/0 conn(0x7f504c106710 legacy=0x7f504c19a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:39.377 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.658+0000 7f5052eba640 1 -- 172.21.15.160:0/1860363171 shutdown_connections 2024-09-16T08:40:39.377 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.658+0000 7f5052eba640 1 -- 172.21.15.160:0/1860363171 >> 172.21.15.160:0/1860363171 conn(0x7f504c102000 msgr2=0x7f504c1043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:39.377 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.658+0000 7f5052eba640 1 -- 172.21.15.160:0/1860363171 shutdown_connections 2024-09-16T08:40:39.377 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:38.658+0000 7f5052eba640 1 -- 172.21.15.160:0/1860363171 wait complete. 2024-09-16T08:40:39.377 INFO:teuthology.orchestra.run.smithi160.stdout:Enabling the dashboard module... 2024-09-16T08:40:39.842 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1860363171' entity='client.admin' 2024-09-16T08:40:40.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:40 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' 2024-09-16T08:40:40.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1879332550' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-16T08:40:40.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:40 smithi160 ceph-mon[30137]: from='mgr.14116 v1:172.21.15.160:0/8907778' entity='mgr.y' 2024-09-16T08:40:40.948 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:40 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ignoring --setuser ceph since I am not root 2024-09-16T08:40:40.948 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:40 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ignoring --setgroup ceph since I am not root 2024-09-16T08:40:41.530 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.924+0000 7fc470b83640 1 Processor -- start 2024-09-16T08:40:41.531 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.924+0000 7fc470b83640 1 -- start start 2024-09-16T08:40:41.531 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.924+0000 7fc470b83640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc46c10aba0 con 0x7fc46c106720 2024-09-16T08:40:41.531 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.924+0000 7fc46a59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fc46c106720 0x7fc46c106b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35294/0 (socket says 172.21.15.160:35294) 2024-09-16T08:40:41.531 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.924+0000 7fc46a59c640 1 -- 172.21.15.160:0/2244008101 learned_addr learned my addr 172.21.15.160:0/2244008101 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:41.531 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.924+0000 7fc46959a640 1 -- 172.21.15.160:0/2244008101 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1648177670 0 0) 0x7fc46c10aba0 con 0x7fc46c106720 2024-09-16T08:40:41.531 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.925+0000 7fc46959a640 1 -- 172.21.15.160:0/2244008101 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc454003520 con 0x7fc46c106720 2024-09-16T08:40:41.531 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.925+0000 7fc46959a640 1 -- 172.21.15.160:0/2244008101 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2473561400 0 0) 0x7fc454003520 con 0x7fc46c106720 2024-09-16T08:40:41.531 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.925+0000 7fc46959a640 1 -- 172.21.15.160:0/2244008101 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46c10bd80 con 0x7fc46c106720 2024-09-16T08:40:41.531 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.925+0000 7fc46959a640 1 -- 172.21.15.160:0/2244008101 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fc45c001600 con 0x7fc46c106720 2024-09-16T08:40:41.531 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.925+0000 7fc46959a640 1 -- 172.21.15.160:0/2244008101 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fc45c001d10 con 0x7fc46c106720 2024-09-16T08:40:41.532 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.925+0000 7fc46959a640 1 -- 172.21.15.160:0/2244008101 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fc45c0058d0 con 0x7fc46c106720 2024-09-16T08:40:41.532 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.925+0000 7fc470b83640 1 -- 172.21.15.160:0/2244008101 >> v1:172.21.15.160:6789/0 conn(0x7fc46c106720 legacy=0x7fc46c106b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:41.532 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.925+0000 7fc470b83640 1 -- 172.21.15.160:0/2244008101 shutdown_connections 2024-09-16T08:40:41.532 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.925+0000 7fc470b83640 1 -- 172.21.15.160:0/2244008101 >> 172.21.15.160:0/2244008101 conn(0x7fc46c102070 msgr2=0x7fc46c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:41.532 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.926+0000 7fc470b83640 1 -- 172.21.15.160:0/2244008101 shutdown_connections 2024-09-16T08:40:41.532 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.926+0000 7fc470b83640 1 -- 172.21.15.160:0/2244008101 wait complete. 2024-09-16T08:40:41.532 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.926+0000 7fc470b83640 1 Processor -- start 2024-09-16T08:40:41.532 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.926+0000 7fc470b83640 1 -- start start 2024-09-16T08:40:41.532 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.926+0000 7fc470b83640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc46c19aa40 con 0x7fc46c106720 2024-09-16T08:40:41.532 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.926+0000 7fc46a59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fc46c106720 0x7fc46c19a2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35310/0 (socket says 172.21.15.160:35310) 2024-09-16T08:40:41.532 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.926+0000 7fc46a59c640 1 -- 172.21.15.160:0/1879332550 learned_addr learned my addr 172.21.15.160:0/1879332550 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:41.532 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.927+0000 7fc45b7fe640 1 -- 172.21.15.160:0/1879332550 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 230950880 0 0) 0x7fc46c19aa40 con 0x7fc46c106720 2024-09-16T08:40:41.532 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.927+0000 7fc45b7fe640 1 -- 172.21.15.160:0/1879332550 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc454005570 con 0x7fc46c106720 2024-09-16T08:40:41.533 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.927+0000 7fc45b7fe640 1 -- 172.21.15.160:0/1879332550 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3660112411 0 0) 0x7fc454005570 con 0x7fc46c106720 2024-09-16T08:40:41.533 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.927+0000 7fc45b7fe640 1 -- 172.21.15.160:0/1879332550 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc46c19aa40 con 0x7fc46c106720 2024-09-16T08:40:41.533 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.927+0000 7fc45b7fe640 1 -- 172.21.15.160:0/1879332550 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fc45c001080 con 0x7fc46c106720 2024-09-16T08:40:41.533 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.927+0000 7fc45b7fe640 1 -- 172.21.15.160:0/1879332550 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1914472431 0 0) 0x7fc46c19aa40 con 0x7fc46c106720 2024-09-16T08:40:41.533 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.927+0000 7fc45b7fe640 1 -- 172.21.15.160:0/1879332550 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46c19ac30 con 0x7fc46c106720 2024-09-16T08:40:41.533 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.927+0000 7fc470b83640 1 -- 172.21.15.160:0/1879332550 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc46c19e510 con 0x7fc46c106720 2024-09-16T08:40:41.533 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.927+0000 7fc470b83640 1 -- 172.21.15.160:0/1879332550 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc46c19e930 con 0x7fc46c106720 2024-09-16T08:40:41.533 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.927+0000 7fc45b7fe640 1 -- 172.21.15.160:0/1879332550 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fc45c0055b0 con 0x7fc46c106720 2024-09-16T08:40:41.533 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.927+0000 7fc45b7fe640 1 -- 172.21.15.160:0/1879332550 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fc45c006030 con 0x7fc46c106720 2024-09-16T08:40:41.533 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.928+0000 7fc45b7fe640 1 -- 172.21.15.160:0/1879332550 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 8) ==== 51730+0+0 (unknown 4165010166 0 0) 0x7fc45c012c90 con 0x7fc46c106720 2024-09-16T08:40:41.533 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.928+0000 7fc470b83640 1 -- 172.21.15.160:0/1879332550 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc44c0050b0 con 0x7fc46c106720 2024-09-16T08:40:41.533 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.928+0000 7fc45b7fe640 1 -- 172.21.15.160:0/1879332550 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 760617087 0 0) 0x7fc45c04f9e0 con 0x7fc46c106720 2024-09-16T08:40:41.533 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:39.933+0000 7fc45b7fe640 1 -- 172.21.15.160:0/1879332550 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fc45c018f10 con 0x7fc46c106720 2024-09-16T08:40:41.534 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:40.168+0000 7fc470b83640 1 -- 172.21.15.160:0/1879332550 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fc44c0053a0 con 0x7fc46c106720 2024-09-16T08:40:41.534 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:40.681+0000 7fc45b7fe640 1 -- 172.21.15.160:0/1879332550 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (unknown 1498667528 0 0) 0x7fc45c018810 con 0x7fc46c106720 2024-09-16T08:40:41.534 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:40.683+0000 7fc45b7fe640 1 -- 172.21.15.160:0/1879332550 <== mon.0 v1:172.21.15.160:6789/0 11 ==== mgrmap(e 9) ==== 51743+0+0 (unknown 3258676433 0 0) 0x7fc45c04e320 con 0x7fc46c106720 2024-09-16T08:40:41.534 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:40.684+0000 7fc470b83640 1 -- 172.21.15.160:0/1879332550 >> v1:172.21.15.160:6800/3425583652 conn(0x7fc454040850 legacy=0x7fc454042d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:41.534 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:40.685+0000 7fc470b83640 1 -- 172.21.15.160:0/1879332550 >> v1:172.21.15.160:6789/0 conn(0x7fc46c106720 legacy=0x7fc46c19a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:41.534 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:40.685+0000 7fc470b83640 1 -- 172.21.15.160:0/1879332550 shutdown_connections 2024-09-16T08:40:41.534 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:40.685+0000 7fc470b83640 1 -- 172.21.15.160:0/1879332550 >> 172.21.15.160:0/1879332550 conn(0x7fc46c102070 msgr2=0x7fc46c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:41.534 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:40.685+0000 7fc470b83640 1 -- 172.21.15.160:0/1879332550 shutdown_connections 2024-09-16T08:40:41.534 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:40.685+0000 7fc470b83640 1 -- 172.21.15.160:0/1879332550 wait complete. 2024-09-16T08:40:41.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1879332550' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-16T08:40:41.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:41 smithi160 ceph-mon[30137]: mgrmap e9: y(active, since 19s) 2024-09-16T08:40:43.442 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/76639540' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-16T08:40:43.725 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:40:43.725 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-16T08:40:43.725 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T08:40:43.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-16T08:40:43.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T08:40:43.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:40:43.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.901+0000 7fdcaf0ff640 1 Processor -- start 2024-09-16T08:40:43.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.901+0000 7fdcaf0ff640 1 -- start start 2024-09-16T08:40:43.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.901+0000 7fdcaf0ff640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fdca8109ae0 con 0x7fdca8105b00 2024-09-16T08:40:43.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.901+0000 7fdcace9b640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fdca8105b00 0x7fdca8105f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35346/0 (socket says 172.21.15.160:35346) 2024-09-16T08:40:43.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.901+0000 7fdcace9b640 1 -- 172.21.15.160:0/1626214155 learned_addr learned my addr 172.21.15.160:0/1626214155 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:43.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.901+0000 7fdca77fe640 1 -- 172.21.15.160:0/1626214155 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4239891317 0 0) 0x7fdca8109ae0 con 0x7fdca8105b00 2024-09-16T08:40:43.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.901+0000 7fdca77fe640 1 -- 172.21.15.160:0/1626214155 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc98003520 con 0x7fdca8105b00 2024-09-16T08:40:43.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.902+0000 7fdca77fe640 1 -- 172.21.15.160:0/1626214155 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1718428981 0 0) 0x7fdc98003520 con 0x7fdca8105b00 2024-09-16T08:40:43.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.902+0000 7fdca77fe640 1 -- 172.21.15.160:0/1626214155 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca810acc0 con 0x7fdca8105b00 2024-09-16T08:40:43.726 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.902+0000 7fdca77fe640 1 -- 172.21.15.160:0/1626214155 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fdc9c001600 con 0x7fdca8105b00 2024-09-16T08:40:43.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.902+0000 7fdca77fe640 1 -- 172.21.15.160:0/1626214155 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fdc9c001c10 con 0x7fdca8105b00 2024-09-16T08:40:43.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.902+0000 7fdca77fe640 1 -- 172.21.15.160:0/1626214155 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fdc9c0058d0 con 0x7fdca8105b00 2024-09-16T08:40:43.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.902+0000 7fdcaf0ff640 1 -- 172.21.15.160:0/1626214155 >> v1:172.21.15.160:6789/0 conn(0x7fdca8105b00 legacy=0x7fdca8105f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:43.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.902+0000 7fdcaf0ff640 1 -- 172.21.15.160:0/1626214155 shutdown_connections 2024-09-16T08:40:43.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.902+0000 7fdcaf0ff640 1 -- 172.21.15.160:0/1626214155 >> 172.21.15.160:0/1626214155 conn(0x7fdca8100680 msgr2=0x7fdca8102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:43.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.903+0000 7fdcaf0ff640 1 -- 172.21.15.160:0/1626214155 shutdown_connections 2024-09-16T08:40:43.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.903+0000 7fdcaf0ff640 1 -- 172.21.15.160:0/1626214155 wait complete. 2024-09-16T08:40:43.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.903+0000 7fdcaf0ff640 1 Processor -- start 2024-09-16T08:40:43.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.903+0000 7fdcaf0ff640 1 -- start start 2024-09-16T08:40:43.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.903+0000 7fdcaf0ff640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fdca819e100 con 0x7fdca8105b00 2024-09-16T08:40:43.727 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.903+0000 7fdcace9b640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fdca8105b00 0x7fdca8105010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35350/0 (socket says 172.21.15.160:35350) 2024-09-16T08:40:43.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.903+0000 7fdcace9b640 1 -- 172.21.15.160:0/76639540 learned_addr learned my addr 172.21.15.160:0/76639540 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:43.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.904+0000 7fdca5ffb640 1 -- 172.21.15.160:0/76639540 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 54551918 0 0) 0x7fdca819e100 con 0x7fdca8105b00 2024-09-16T08:40:43.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.904+0000 7fdca5ffb640 1 -- 172.21.15.160:0/76639540 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc980055c0 con 0x7fdca8105b00 2024-09-16T08:40:43.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.904+0000 7fdca5ffb640 1 -- 172.21.15.160:0/76639540 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1476249281 0 0) 0x7fdc980055c0 con 0x7fdca8105b00 2024-09-16T08:40:43.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.904+0000 7fdca5ffb640 1 -- 172.21.15.160:0/76639540 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdca819e100 con 0x7fdca8105b00 2024-09-16T08:40:43.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.904+0000 7fdca5ffb640 1 -- 172.21.15.160:0/76639540 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fdc9c001080 con 0x7fdca8105b00 2024-09-16T08:40:43.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.904+0000 7fdca5ffb640 1 -- 172.21.15.160:0/76639540 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3430962050 0 0) 0x7fdca819e100 con 0x7fdca8105b00 2024-09-16T08:40:43.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.904+0000 7fdca5ffb640 1 -- 172.21.15.160:0/76639540 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca819f2e0 con 0x7fdca8105b00 2024-09-16T08:40:43.728 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.904+0000 7fdcaf0ff640 1 -- 172.21.15.160:0/76639540 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fdca819e330 con 0x7fdca8105b00 2024-09-16T08:40:43.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.904+0000 7fdca5ffb640 1 -- 172.21.15.160:0/76639540 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fdc9c004bf0 con 0x7fdca8105b00 2024-09-16T08:40:43.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.904+0000 7fdca5ffb640 1 -- 172.21.15.160:0/76639540 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fdc9c0061d0 con 0x7fdca8105b00 2024-09-16T08:40:43.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.904+0000 7fdcaf0ff640 1 -- 172.21.15.160:0/76639540 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fdca819e870 con 0x7fdca8105b00 2024-09-16T08:40:43.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.905+0000 7fdca5ffb640 1 -- 172.21.15.160:0/76639540 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 9) ==== 51743+0+0 (unknown 3258676433 0 0) 0x7fdc9c012e40 con 0x7fdca8105b00 2024-09-16T08:40:43.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.905+0000 7fdca5ffb640 1 -- 172.21.15.160:0/76639540 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 760617087 0 0) 0x7fdc9c050470 con 0x7fdca8105b00 2024-09-16T08:40:43.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.905+0000 7fdca7fff640 1 -- 172.21.15.160:0/76639540 >> v1:172.21.15.160:6800/3425583652 conn(0x7fdc98040910 legacy=0x7fdc98042dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/3425583652 2024-09-16T08:40:43.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.905+0000 7fdcaf0ff640 1 -- 172.21.15.160:0/76639540 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca81925d0 con 0x7fdca8105b00 2024-09-16T08:40:43.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:42.910+0000 7fdca5ffb640 1 -- 172.21.15.160:0/76639540 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fdc9c018d90 con 0x7fdca8105b00 2024-09-16T08:40:43.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:43.106+0000 7fdca7fff640 1 -- 172.21.15.160:0/76639540 >> v1:172.21.15.160:6800/3425583652 conn(0x7fdc98040910 legacy=0x7fdc98042dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/3425583652 2024-09-16T08:40:43.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:43.140+0000 7fdcaf0ff640 1 -- 172.21.15.160:0/76639540 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fdca819ec00 con 0x7fdca8105b00 2024-09-16T08:40:43.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:43.141+0000 7fdca5ffb640 1 -- 172.21.15.160:0/76639540 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (unknown 2005831594 0 1748205903) 0x7fdc9c018690 con 0x7fdca8105b00 2024-09-16T08:40:43.729 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:43.143+0000 7fdcaf0ff640 1 -- 172.21.15.160:0/76639540 >> v1:172.21.15.160:6800/3425583652 conn(0x7fdc98040910 legacy=0x7fdc98042dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T08:40:43.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:43.143+0000 7fdcaf0ff640 1 -- 172.21.15.160:0/76639540 >> v1:172.21.15.160:6789/0 conn(0x7fdca8105b00 legacy=0x7fdca8105010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:43.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:43.143+0000 7fdcaf0ff640 1 -- 172.21.15.160:0/76639540 shutdown_connections 2024-09-16T08:40:43.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:43.143+0000 7fdcaf0ff640 1 -- 172.21.15.160:0/76639540 >> 172.21.15.160:0/76639540 conn(0x7fdca8100680 msgr2=0x7fdca8102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:43.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:43.143+0000 7fdcaf0ff640 1 -- 172.21.15.160:0/76639540 shutdown_connections 2024-09-16T08:40:43.730 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:43.143+0000 7fdcaf0ff640 1 -- 172.21.15.160:0/76639540 wait complete. 2024-09-16T08:40:43.730 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for the mgr to restart... 2024-09-16T08:40:43.730 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mgr epoch 9... 2024-09-16T08:40:45.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:45 smithi160 ceph-mon[30137]: Active manager daemon y restarted 2024-09-16T08:40:45.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:45 smithi160 ceph-mon[30137]: Activating manager daemon y 2024-09-16T08:40:45.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:45 smithi160 ceph-mon[30137]: osdmap e3: 0 total, 0 up, 0 in 2024-09-16T08:40:45.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:45 smithi160 ceph-mon[30137]: mgrmap e10: y(active, starting, since 0.00352157s) 2024-09-16T08:40:45.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:45 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:40:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:45 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-16T08:40:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:45 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-16T08:40:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:45 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-16T08:40:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:45 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-16T08:40:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:45 smithi160 ceph-mon[30137]: Manager daemon y is now available 2024-09-16T08:40:47.150 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:46 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:40:47.150 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:46 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-16T08:40:47.150 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:46 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-16T08:40:47.150 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:46 smithi160 ceph-mon[30137]: mgrmap e11: y(active, since 1.00626s) 2024-09-16T08:40:47.150 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: 2024-09-16T08:40:47.036+0000 7fdcabebf640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:08:40:47] ENGINE Error in HTTPServer.serve 2024-09-16T08:40:47.151 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: Traceback (most recent call last): 2024-09-16T08:40:47.151 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:40:47.151 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: self._connections.run(self.expiration_interval) 2024-09-16T08:40:47.151 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:40:47.151 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: self._run(expiration_interval) 2024-09-16T08:40:47.151 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:40:47.151 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:40:47.151 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:40:47.151 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:40:47.151 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:40:47.151 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: s = self.context.wrap_socket( 2024-09-16T08:40:47.152 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:40:47.152 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: return self.sslsocket_class._create( 2024-09-16T08:40:47.152 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:40:47.152 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: self.do_handshake() 2024-09-16T08:40:47.152 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:40:47.152 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: self._sslobj.do_handshake() 2024-09-16T08:40:47.152 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:40:47.152 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: 2024-09-16T08:40:47.576 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:40:47.576 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-16T08:40:47.576 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T08:40:47.576 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:40:47.576 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.225+0000 7f84d1416640 1 Processor -- start 2024-09-16T08:40:47.576 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.225+0000 7f84d1416640 1 -- start start 2024-09-16T08:40:47.576 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.225+0000 7f84d1416640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f84cc10aaf0 con 0x7f84cc07a420 2024-09-16T08:40:47.576 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.225+0000 7f84caffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f84cc07a420 0x7f84cc109340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35364/0 (socket says 172.21.15.160:35364) 2024-09-16T08:40:47.576 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.225+0000 7f84caffd640 1 -- 172.21.15.160:0/176660856 learned_addr learned my addr 172.21.15.160:0/176660856 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:47.577 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.225+0000 7f84c9ffb640 1 -- 172.21.15.160:0/176660856 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2741265519 0 0) 0x7f84cc10aaf0 con 0x7f84cc07a420 2024-09-16T08:40:47.577 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.225+0000 7f84c9ffb640 1 -- 172.21.15.160:0/176660856 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84b4003520 con 0x7f84cc07a420 2024-09-16T08:40:47.577 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.226+0000 7f84c9ffb640 1 -- 172.21.15.160:0/176660856 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4070507596 0 0) 0x7f84b4003520 con 0x7f84cc07a420 2024-09-16T08:40:47.577 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.226+0000 7f84c9ffb640 1 -- 172.21.15.160:0/176660856 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84cc10bcd0 con 0x7f84cc07a420 2024-09-16T08:40:47.577 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.226+0000 7f84c9ffb640 1 -- 172.21.15.160:0/176660856 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f84bc001600 con 0x7f84cc07a420 2024-09-16T08:40:47.577 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.226+0000 7f84c9ffb640 1 -- 172.21.15.160:0/176660856 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f84bc001c10 con 0x7f84cc07a420 2024-09-16T08:40:47.577 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.226+0000 7f84c9ffb640 1 -- 172.21.15.160:0/176660856 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f84bc0058d0 con 0x7f84cc07a420 2024-09-16T08:40:47.577 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.226+0000 7f84d1416640 1 -- 172.21.15.160:0/176660856 >> v1:172.21.15.160:6789/0 conn(0x7f84cc07a420 legacy=0x7f84cc109340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:47.577 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.226+0000 7f84d1416640 1 -- 172.21.15.160:0/176660856 shutdown_connections 2024-09-16T08:40:47.577 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.226+0000 7f84d1416640 1 -- 172.21.15.160:0/176660856 >> 172.21.15.160:0/176660856 conn(0x7f84cc100f90 msgr2=0x7f84cc1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:47.577 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.226+0000 7f84d1416640 1 -- 172.21.15.160:0/176660856 shutdown_connections 2024-09-16T08:40:47.577 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.226+0000 7f84d1416640 1 -- 172.21.15.160:0/176660856 wait complete. 2024-09-16T08:40:47.577 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.227+0000 7f84d1416640 1 Processor -- start 2024-09-16T08:40:47.578 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.227+0000 7f84d1416640 1 -- start start 2024-09-16T08:40:47.578 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.227+0000 7f84d1416640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f84cc1070f0 con 0x7f84cc07a420 2024-09-16T08:40:47.578 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.227+0000 7f84caffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f84cc07a420 0x7f84cc106990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35366/0 (socket says 172.21.15.160:35366) 2024-09-16T08:40:47.578 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.227+0000 7f84caffd640 1 -- 172.21.15.160:0/1038304987 learned_addr learned my addr 172.21.15.160:0/1038304987 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:47.578 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.228+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1197148452 0 0) 0x7f84cc1070f0 con 0x7f84cc07a420 2024-09-16T08:40:47.578 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.228+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84b40055c0 con 0x7f84cc07a420 2024-09-16T08:40:47.578 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.228+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2958025826 0 0) 0x7f84b40055c0 con 0x7f84cc07a420 2024-09-16T08:40:47.578 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.228+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f84cc1070f0 con 0x7f84cc07a420 2024-09-16T08:40:47.578 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.228+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f84bc005220 con 0x7f84cc07a420 2024-09-16T08:40:47.578 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.228+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2588503122 0 0) 0x7f84cc1070f0 con 0x7f84cc07a420 2024-09-16T08:40:47.578 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.228+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84cc1a1490 con 0x7f84cc07a420 2024-09-16T08:40:47.578 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.228+0000 7f84d1416640 1 -- 172.21.15.160:0/1038304987 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f84cc1a04e0 con 0x7f84cc07a420 2024-09-16T08:40:47.578 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.228+0000 7f84d1416640 1 -- 172.21.15.160:0/1038304987 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f84cc1a0a20 con 0x7f84cc07a420 2024-09-16T08:40:47.579 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.228+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f84bc001080 con 0x7f84cc07a420 2024-09-16T08:40:47.579 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.228+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f84bc005e70 con 0x7f84cc07a420 2024-09-16T08:40:47.579 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.229+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 9) ==== 51743+0+0 (unknown 3258676433 0 0) 0x7f84bc012a40 con 0x7f84cc07a420 2024-09-16T08:40:47.579 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.229+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 --> v1:172.21.15.160:6800/3425583652 -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f84b4044620 con 0x7f84b4040950 2024-09-16T08:40:47.579 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.229+0000 7f84ca7fc640 1 -- 172.21.15.160:0/1038304987 >> v1:172.21.15.160:6800/3425583652 conn(0x7f84b4040950 legacy=0x7f84b4042e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/3425583652 2024-09-16T08:40:47.579 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.230+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 760617087 0 0) 0x7f84bc04f670 con 0x7f84cc07a420 2024-09-16T08:40:47.579 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.431+0000 7f84ca7fc640 1 -- 172.21.15.160:0/1038304987 >> v1:172.21.15.160:6800/3425583652 conn(0x7f84b4040950 legacy=0x7f84b4042e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/3425583652 2024-09-16T08:40:47.579 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:44.833+0000 7f84ca7fc640 1 -- 172.21.15.160:0/1038304987 >> v1:172.21.15.160:6800/3425583652 conn(0x7f84b4040950 legacy=0x7f84b4042e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/3425583652 2024-09-16T08:40:47.579 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:45.635+0000 7f84ca7fc640 1 -- 172.21.15.160:0/1038304987 >> v1:172.21.15.160:6800/3425583652 conn(0x7f84b4040950 legacy=0x7f84b4042e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/3425583652 2024-09-16T08:40:47.579 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:45.805+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mgrmap(e 10) ==== 51545+0+0 (unknown 460808886 0 0) 0x7f84bc004bf0 con 0x7f84cc07a420 2024-09-16T08:40:47.579 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:45.805+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 >> v1:172.21.15.160:6800/3425583652 conn(0x7f84b4040950 legacy=0x7f84b4042e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T08:40:47.579 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:46.806+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mgrmap(e 11) ==== 51637+0+0 (unknown 3974579090 0 0) 0x7f84bc051020 con 0x7f84cc07a420 2024-09-16T08:40:47.580 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:46.807+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 --> v1:172.21.15.160:6800/1286008384 -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f84b4044620 con 0x7f84b40453f0 2024-09-16T08:40:47.580 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:46.808+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (unknown 0 0 1361822222) 0x7f84b4044620 con 0x7f84b40453f0 2024-09-16T08:40:47.580 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:46.815+0000 7f84d1416640 1 -- 172.21.15.160:0/1038304987 --> v1:172.21.15.160:6800/1286008384 -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f84b80025a0 con 0x7f84b40453f0 2024-09-16T08:40:47.580 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:46.815+0000 7f84b3fff640 1 -- 172.21.15.160:0/1038304987 <== mgr.14150 v1:172.21.15.160:6800/1286008384 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (unknown 0 0 3086460295) 0x7f84b80025a0 con 0x7f84b40453f0 2024-09-16T08:40:47.580 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:46.815+0000 7f84d1416640 1 -- 172.21.15.160:0/1038304987 >> v1:172.21.15.160:6800/1286008384 conn(0x7f84b40453f0 legacy=0x7f84b40477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:47.580 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:46.815+0000 7f84d1416640 1 -- 172.21.15.160:0/1038304987 >> v1:172.21.15.160:6789/0 conn(0x7f84cc07a420 legacy=0x7f84cc106990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:47.580 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:46.815+0000 7f84d1416640 1 -- 172.21.15.160:0/1038304987 shutdown_connections 2024-09-16T08:40:47.580 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:46.815+0000 7f84d1416640 1 -- 172.21.15.160:0/1038304987 >> 172.21.15.160:0/1038304987 conn(0x7f84cc100f90 msgr2=0x7f84cc191040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:47.580 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:46.816+0000 7f84d1416640 1 -- 172.21.15.160:0/1038304987 shutdown_connections 2024-09-16T08:40:47.580 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:46.816+0000 7f84d1416640 1 -- 172.21.15.160:0/1038304987 wait complete. 2024-09-16T08:40:47.580 INFO:teuthology.orchestra.run.smithi160.stdout:mgr epoch 9 is available 2024-09-16T08:40:47.580 INFO:teuthology.orchestra.run.smithi160.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-16T08:40:48.196 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-mon[30137]: [16/Sep/2024:08:40:46] ENGINE Bus STARTING 2024-09-16T08:40:48.196 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-mon[30137]: [16/Sep/2024:08:40:47] ENGINE Serving on https://172.21.15.160:7150 2024-09-16T08:40:48.196 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-mon[30137]: [16/Sep/2024:08:40:47] ENGINE Error in HTTPServer.serve 2024-09-16T08:40:48.197 INFO:journalctl@ceph.mon.a.smithi160.stdout: Traceback (most recent call last): 2024-09-16T08:40:48.197 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:40:48.197 INFO:journalctl@ceph.mon.a.smithi160.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:40:48.197 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:40:48.197 INFO:journalctl@ceph.mon.a.smithi160.stdout: self._run(expiration_interval) 2024-09-16T08:40:48.197 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:40:48.197 INFO:journalctl@ceph.mon.a.smithi160.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:40:48.197 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:40:48.197 INFO:journalctl@ceph.mon.a.smithi160.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:40:48.197 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:40:48.197 INFO:journalctl@ceph.mon.a.smithi160.stdout: s = self.context.wrap_socket( 2024-09-16T08:40:48.197 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:40:48.198 INFO:journalctl@ceph.mon.a.smithi160.stdout: return self.sslsocket_class._create( 2024-09-16T08:40:48.198 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:40:48.198 INFO:journalctl@ceph.mon.a.smithi160.stdout: self.do_handshake() 2024-09-16T08:40:48.198 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:40:48.198 INFO:journalctl@ceph.mon.a.smithi160.stdout: self._sslobj.do_handshake() 2024-09-16T08:40:48.198 INFO:journalctl@ceph.mon.a.smithi160.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:40:48.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-mon[30137]: [16/Sep/2024:08:40:47] ENGINE Serving on http://172.21.15.160:8765 2024-09-16T08:40:48.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:47 smithi160 ceph-mon[30137]: [16/Sep/2024:08:40:47] ENGINE Bus STARTED 2024-09-16T08:40:48.935 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:48 smithi160 ceph-mon[30137]: mgrmap e12: y(active, since 2s) 2024-09-16T08:40:48.935 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:48 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:40:48.935 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:48 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:40:48.935 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:48 smithi160 ceph-mon[30137]: from='client.14162 v1:172.21.15.160:0/3986927599' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:40:50.222 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIFETCCAvmgAwIBAgIUOoNmjbsQ2gFkyKVeYGcT1CsSL14wDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkxNjA4NDA0OVoXDTM0\nMDkxNzA4NDA0OVowGDEWMBQGA1UEAwwNMTcyLjIxLjE1LjE2MDCCAiIwDQYJKoZI\nhvcNAQEBBQADggIPADCCAgoCggIBAIxI7ik4GBoaSeTmY6Q+NtzogAJaPxh2x21k\nVz6+y0ATLBcffHlc9HK5EWugEluiORd7oZ1yrQOyG5aubVy78TcMvQitaCdEOYje\n0ScgStp3hQgzduqOyk6/yN1mWRLIiWnrdvpSDZohN2vn6aaZje2E6IU/wk/+sDup\npaFmhP1y8UceF9H/3W9CjY/Dl8bGgRJ0YISd5xxgmNXBQTm3vZK+VYFCRnyFMJlE\n8R+rZm2MYBCAw4jZs/iQbt3bBIGuH9i70AbLXVBq+wpd2O00i4OUDU1q+9xxu2yX\n9qXA1CDIbhsQoojt4N+VKb8KR2f15Lk8GkFYrOp/3bVl0yQ/BQeV4yIxIXNsWcL/\npjgvRyE3m91JzlWn9KwUUS8ZEOo6jyN4ZQLZD07qUCr1m/xRm6x84WiTkAwbHlnj\n/LA6+ehplMLI3FBl/QFfyALR37dJTPkxK5x6OCDeV2u5imMrk2ox9yVGhvXNWC7W\n0S3smAV5AVgdjPY9S90wvjae8jm1z078ak4XT8MB7+ZBq1QWcD0tm874VXRn4rtn\nHA+WCMjOeIW/PD4NfxD1fZFjnsHllsz+uwV/uk+SjP+yG8F3iOx+nt9y+ByQiNvv\n3nlLwWxfL9xAUFNQRFALlo8oJU7jWvb0pLtHyZ1u+tJb54DvVbHUYCK5l4Pu2BYK\nbubmIRRFAgMBAAGjVDBSMEIGA1UdEQQ7MDmCHnNtaXRoaTE2MC5mcm9udC5zZXBp\nYS5jZXBoLmNvbYIRZGFzaGJvYXJkX3NlcnZlcnOHBKwVD6AwDAYDVR0TAQH/BAIw\nADANBgkqhkiG9w0BAQsFAAOCAgEAlh4s63Y6E3rEzknhUeNq0zVeItQhdi9C/hCe\nWWNxRCrgKMdUu1Qdl4yH8f0WFxsfmkudBPeU+4SFyDxQ785j+sB680dVQRzH80sb\nHQVKUrU7qXCsRl1eWuSaacRAEm5PhozW9Old9xAZdiRadFnQ5sgTR55yVzaBG5gn\netxgBTsDCnTFd86lRj2j6MuGg5ZkKK36GKn447PNSAIy1eq4k/UDygsgHhFQCMOk\nc3YCVDtNAOO0fzhTNEnODj70+PWJNG+FywR+4QQqBpGZpOIEY4UIVPFFYmca9+Sn\n9iOFdgk2uzbb9xiR+Un/3uB1NuIRmciX0fmRj4+YK/SLhg24LG9QxD8n6vHL1Gam\nWKiBraZwa+3WK43qa+R8x4vZJd7Wj9kLwC5Ps4jbQOdYPQmpNvkpkWrk2rJ2TtWt\n1Ayiv+xsVk9WCe7p4R69Yd/F1ksD4nmvXC6IiEAROWvj7RRDvpf+Eug5gRgAhuyP\nAaBfrZbSL8vKHQiKKfVDRUPuHNMAADG/vEC6u64B9jjgalCfEw3AfInHQP96Y67K\n99qSpyE5A0jKcvf7c75BO+Gn03VlJXUY1AZYdpYWZt16A4k3+weAkR8+WJNkK6kG\n6corxbTChI/IJqsapVFQC/QeUznPgyOD0WBnyCMWYdoKfkpk+b3Cy0yA3R5JoTfw\n/2edAqg=\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJJwIBAAKCAgEAjEjuKTgYGhpJ5OZjpD423OiAAlo/GHbHbWRXPr7LQBMsFx98\neVz0crkRa6ASW6I5F3uhnXKtA7Iblq5tXLvxNwy9CK1oJ0Q5iN7RJyBK2neFCDN2\n6o7KTr/I3WZZEsiJaet2+lINmiE3a+fpppmN7YTohT/CT/6wO6mloWaE/XLxRx4X\n0f/db0KNj8OXxsaBEnRghJ3nHGCY1cFBObe9kr5VgUJGfIUwmUTxH6tmbYxgEIDD\niNmz+JBu3dsEga4f2LvQBstdUGr7Cl3Y7TSLg5QNTWr73HG7bJf2pcDUIMhuGxCi\niO3g35UpvwpHZ/XkuTwaQVis6n/dtWXTJD8FB5XjIjEhc2xZwv+mOC9HITeb3UnO\nVaf0rBRRLxkQ6jqPI3hlAtkPTupQKvWb/FGbrHzhaJOQDBseWeP8sDr56GmUwsjc\nUGX9AV/IAtHft0lM+TErnHo4IN5Xa7mKYyuTajH3JUaG9c1YLtbRLeyYBXkBWB2M\n9j1L3TC+Np7yObXPTvxqThdPwwHv5kGrVBZwPS2bzvhVdGfiu2ccD5YIyM54hb88\nPg1/EPV9kWOeweWWzP67BX+6T5KM/7IbwXeI7H6e33L4HJCI2+/eeUvBbF8v3EBQ\nU1BEUAuWjyglTuNa9vSku0fJnW760lvngO9VsdRgIrmXg+7YFgpu5uYhFEUCAwEA\nAQKCAgABe5ze3SuuDPSWgZPwjsLMM3306GjN1KWXg63DaUdjOkCnTuslKkUgOFIV\nkefXxZ92XtVuwWFQTm/puBbKWec7FnyiDJky/w7fokFSOU5pQEWs+ERLxu4fBnp0\nUqpi3zFKqBVGIJtps2uvTZCD9RARwGunyux+JXpN6yVnzxpdRnuD7a7BRsPbsvel\nmyLt1DfVjtrdVAD3JRNAgS40ErvjgvUK7H0rwEmq+RZWaBypUEhMrFvExM5Y9Ifj\nevbxevp1s0iH5drq6HKWYbcTDNDjt2srlBOs+RvK0lFmX6MuZIf7a8HYEjGkE2LR\nVOW71NbR2W+KnMoZIXhhvD2O3saqEZqIxPpmfTzC58Qa9NlTR2S/TqrYJLcBiZED\npkVH6hk66oCQAhMdMkiZM6oi/EX2urKN7eaVSpwI7zRPP9NJ7DIjIwfOsFcpwylo\nb1oqSFJ92Yvmn54e8/OZssOuAIRLGOSzAx6qsr/cd3kJl71u1e5JxLn4HvvjevP6\nBvgAzaQhN7Vw+4Aq5vMbumzro4EG010lHgHK4asvoLmtb92neymzbZ8tU1zZFyON\nHsdtxi/CXlYU4UFEAcFf3v3926Nn/P3wx/tVVeSyGcpeTbVPNgPK0tdJzxlnDg9H\nmFa0PiUKiHlRwZdAWG6sGFq+Ykt5JwtUKB8Ki/ohzBftvZT12QKCAQEAxDIYpZgv\nFq2QTesPJ7XcFtn3MIkGex3WXHAePnM/E45xeQ9+7fIMb8ePQCu88a6I5iyD6qpn\nbSL//Tsvgb5QgBU4zUmJLxZOQfZjHX4/Yvx2KJWHSZmpNQTrNzDJbppgGRhsV0Ua\nOMx32z9KWU1AkdUSQBnc0Reth7Jhrz0EWJZNqs7GLiV2brZv61aofk7UgepaLMOR\nqMy3DORNN9FSg30vtKTUt6kLKbbuM+0wHCBRDKEe3kOzvSz2MEzkqe3LI5jXHTH+\nCyOcC2O0+6cHUoKNqER170a2NRj0LUqtEmxyNR6p6PAyzQjK6zjDt4eAbuIJtaEb\no3s1KofOa5MbmQKCAQEAtwvmKCi8eiootN5G4xBAtHjXEkkqwOC4IM9XJXlK7mjO\n66nzp3FqPgEtNgGbVNvcq2N0bZS8KqyHuoFni3zpH+QdvMWZ+7ygVaYqvay2cwIy\npObzF8T3T7BPaP01hRAX+nVAztsxQKNJGvq44hZb0Zn0AZOs4xmvTZm+Os5UQoxL\n7vk7QJlKON4epqavUU1a2nN4/R8vKdvMB7J6wNm/WE93Lk5C+wyjx6ib27J5Qx9A\nga/DNxLrpZA3Mdpfqk88L1oAmndhyUmJQGoaS6B5zw4AeqBdd3RaA3N6ap9KYg6C\ntckN/EVwSultVNd8Qb5SdsaLxtKXlsaO8BQ9tMaJjQKCAQBN1pZCtJMSC0AyOazk\nk1lDU6yne+WvWNIgEHdfTXfb+Wp5njU8kAhxHJpv2ip3NEiqEUUxB/RKr1UikaWy\ncuAbQNAtWhxsKwxXTTpbjCf3zLPQPCrVf3pLR61OUzPHhsXTjwYExay2liCDQqPr\ntCOG+CLqfZhG+H7nVFU/IhUkVFVyC+FCk6heGlFi7BmjKwFSnbizA9A5SSbinX0p\nRQnYkwUnRTIDv7RnE2WEAnlQ311ohk/p3LBlwH3YAdnCtU4fzFakBZ6FqtYRCrjW\nEplsGdG7zZWAnexSOpk9kuABbDcPbh/5A7sT74YaNoYnvUGuPE2f2tBNVudr/T9f\nHJkZAoIBABTxLZHmpzmq3SFkVOPhliWPtbuH13Zdv3MxhZzp9aK1JwdbWF8N4MPb\nSprlMCpTRTnulQF8hNK1DopOB9Szo9BcPvK8ANQGTqm96yHsQquB0t1+9sz/YfQi\nDdPLNHZW0wd6psGD3mywwcpuaPyk5jhizsHnbV5GMAJG1rbPqGA8wOtEZH0NjKpY\n71TNuT0qBnz7vRnomXLGjN9MkjLCNtwZq18G4S7r0Ukkh0BpeOovByhPPZxtc7xr\ndYUkoNFRY2dferYxdmSbHzLZcMy6PXVs+q0M7+Sc0TeX3KeaNd5mHppBuHHmV+Sa\nNEICrVqwNasVKCTFL0a2zN+20SB/nnkCggEAAdV1oBhiiv4PVGjHis3HN56B76oL\niwCgs1j+Hmmr14Rci5W5lYedVo75NeU3X7HeSU+f6VXGO8mXIc5J+ciNKIDfTn1Y\nCLq+qPnXgL33zOQzDuGNVR45bPfC9diH8M/ePmYx8Od4M9tgOe7bSIHntBdp1Daf\nhGy9IieC4RF0SAu/S5mAIBgb1DAr0tRhrf7Kle2jkWM5cA1rmdkR3f3VklOJUM06\nTMpsib60VdFx+CzzPze/nHFgEQCZ3zneWOkuXdby9lzxmgvWIzjVtTyd08DkO8C+\n42rYDMrJCvktviHtu2Kd6XvvCSTR68AH3Svg91e80icwfc9ZyxKP3QBwqQ==\n-----END RSA PRIVATE KEY-----\n"} 2024-09-16T08:40:50.222 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.431+0000 7f8ce116a640 1 Processor -- start 2024-09-16T08:40:50.222 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.431+0000 7f8ce116a640 1 -- start start 2024-09-16T08:40:50.222 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.431+0000 7f8ce116a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f8cdc109ad0 con 0x7f8cdc105650 2024-09-16T08:40:50.223 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.431+0000 7f8cdad9d640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f8cdc105650 0x7f8cdc105a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:40116/0 (socket says 172.21.15.160:40116) 2024-09-16T08:40:50.223 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.431+0000 7f8cdad9d640 1 -- 172.21.15.160:0/345727163 learned_addr learned my addr 172.21.15.160:0/345727163 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:50.223 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.431+0000 7f8cd9d9b640 1 -- 172.21.15.160:0/345727163 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1836624335 0 0) 0x7f8cdc109ad0 con 0x7f8cdc105650 2024-09-16T08:40:50.223 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.431+0000 7f8cd9d9b640 1 -- 172.21.15.160:0/345727163 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cc4005730 con 0x7f8cdc105650 2024-09-16T08:40:50.223 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.432+0000 7f8cd9d9b640 1 -- 172.21.15.160:0/345727163 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1603149936 0 0) 0x7f8cc4005730 con 0x7f8cdc105650 2024-09-16T08:40:50.223 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.432+0000 7f8cd9d9b640 1 -- 172.21.15.160:0/345727163 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc10acb0 con 0x7f8cdc105650 2024-09-16T08:40:50.223 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.432+0000 7f8cd9d9b640 1 -- 172.21.15.160:0/345727163 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f8ccc001600 con 0x7f8cdc105650 2024-09-16T08:40:50.223 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.432+0000 7f8cd9d9b640 1 -- 172.21.15.160:0/345727163 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f8ccc001d10 con 0x7f8cdc105650 2024-09-16T08:40:50.223 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.432+0000 7f8cd9d9b640 1 -- 172.21.15.160:0/345727163 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f8ccc0058d0 con 0x7f8cdc105650 2024-09-16T08:40:50.224 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.432+0000 7f8ce116a640 1 -- 172.21.15.160:0/345727163 >> v1:172.21.15.160:6789/0 conn(0x7f8cdc105650 legacy=0x7f8cdc105a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:50.224 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.432+0000 7f8ce116a640 1 -- 172.21.15.160:0/345727163 shutdown_connections 2024-09-16T08:40:50.224 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.432+0000 7f8ce116a640 1 -- 172.21.15.160:0/345727163 >> 172.21.15.160:0/345727163 conn(0x7f8cdc100f80 msgr2=0x7f8cdc1033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:50.224 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.432+0000 7f8ce116a640 1 -- 172.21.15.160:0/345727163 shutdown_connections 2024-09-16T08:40:50.224 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.432+0000 7f8ce116a640 1 -- 172.21.15.160:0/345727163 wait complete. 2024-09-16T08:40:50.224 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.433+0000 7f8ce116a640 1 Processor -- start 2024-09-16T08:40:50.224 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.433+0000 7f8ce116a640 1 -- start start 2024-09-16T08:40:50.224 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.433+0000 7f8ce116a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f8cdc198890 con 0x7f8cdc105650 2024-09-16T08:40:50.224 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.434+0000 7f8cdad9d640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f8cdc105650 0x7f8cdc198130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:40128/0 (socket says 172.21.15.160:40128) 2024-09-16T08:40:50.224 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.434+0000 7f8cdad9d640 1 -- 172.21.15.160:0/3986927599 learned_addr learned my addr 172.21.15.160:0/3986927599 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:50.224 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.434+0000 7f8cc3fff640 1 -- 172.21.15.160:0/3986927599 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3242413496 0 0) 0x7f8cdc198890 con 0x7f8cdc105650 2024-09-16T08:40:50.224 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.434+0000 7f8cc3fff640 1 -- 172.21.15.160:0/3986927599 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cc4007780 con 0x7f8cdc105650 2024-09-16T08:40:50.225 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.434+0000 7f8cc3fff640 1 -- 172.21.15.160:0/3986927599 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1937203253 0 0) 0x7f8cc4007780 con 0x7f8cdc105650 2024-09-16T08:40:50.225 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.434+0000 7f8cc3fff640 1 -- 172.21.15.160:0/3986927599 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8cdc198890 con 0x7f8cdc105650 2024-09-16T08:40:50.225 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.434+0000 7f8cc3fff640 1 -- 172.21.15.160:0/3986927599 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f8ccc001080 con 0x7f8cdc105650 2024-09-16T08:40:50.225 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.434+0000 7f8cc3fff640 1 -- 172.21.15.160:0/3986927599 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 497423140 0 0) 0x7f8cdc198890 con 0x7f8cdc105650 2024-09-16T08:40:50.225 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.434+0000 7f8cc3fff640 1 -- 172.21.15.160:0/3986927599 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc198a80 con 0x7f8cdc105650 2024-09-16T08:40:50.225 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.434+0000 7f8ce116a640 1 -- 172.21.15.160:0/3986927599 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8cdc19c360 con 0x7f8cdc105650 2024-09-16T08:40:50.225 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.434+0000 7f8ce116a640 1 -- 172.21.15.160:0/3986927599 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f8cdc19c780 con 0x7f8cdc105650 2024-09-16T08:40:50.225 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.434+0000 7f8cc3fff640 1 -- 172.21.15.160:0/3986927599 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f8ccc0057d0 con 0x7f8cdc105650 2024-09-16T08:40:50.226 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.434+0000 7f8cc3fff640 1 -- 172.21.15.160:0/3986927599 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f8ccc0060c0 con 0x7f8cdc105650 2024-09-16T08:40:50.226 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.435+0000 7f8cc3fff640 1 -- 172.21.15.160:0/3986927599 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 12) ==== 51743+0+0 (unknown 4227151905 0 0) 0x7f8ccc007390 con 0x7f8cdc105650 2024-09-16T08:40:50.226 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.435+0000 7f8cc3fff640 1 -- 172.21.15.160:0/3986927599 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 2761826324 0 0) 0x7f8ccc0505b0 con 0x7f8cdc105650 2024-09-16T08:40:50.226 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.436+0000 7f8ce116a640 1 -- 172.21.15.160:0/3986927599 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cc80050b0 con 0x7f8cdc105650 2024-09-16T08:40:50.226 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.440+0000 7f8cc3fff640 1 -- 172.21.15.160:0/3986927599 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f8ccc018e90 con 0x7f8cdc105650 2024-09-16T08:40:50.226 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:48.636+0000 7f8ce116a640 1 -- 172.21.15.160:0/3986927599 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}) -- 0x7f8cc8000bc0 con 0x7f8cc4042b00 2024-09-16T08:40:50.226 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:49.270+0000 7f8cc3fff640 1 -- 172.21.15.160:0/3986927599 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+5166 (unknown 0 0 2361141088) 0x7f8cc8000bc0 con 0x7f8cc4042b00 2024-09-16T08:40:50.226 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:49.272+0000 7f8ce116a640 1 -- 172.21.15.160:0/3986927599 >> v1:172.21.15.160:6800/1286008384 conn(0x7f8cc4042b00 legacy=0x7f8cc4044fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:50.226 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:49.272+0000 7f8ce116a640 1 -- 172.21.15.160:0/3986927599 >> v1:172.21.15.160:6789/0 conn(0x7f8cdc105650 legacy=0x7f8cdc198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:50.226 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:49.272+0000 7f8ce116a640 1 -- 172.21.15.160:0/3986927599 shutdown_connections 2024-09-16T08:40:50.227 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:49.272+0000 7f8ce116a640 1 -- 172.21.15.160:0/3986927599 >> 172.21.15.160:0/3986927599 conn(0x7f8cdc100f80 msgr2=0x7f8cdc103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:50.227 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:49.272+0000 7f8ce116a640 1 -- 172.21.15.160:0/3986927599 shutdown_connections 2024-09-16T08:40:50.227 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:49.272+0000 7f8ce116a640 1 -- 172.21.15.160:0/3986927599 wait complete. 2024-09-16T08:40:51.594 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-16T08:40:51.594 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.794+0000 7f37f4852640 1 Processor -- start 2024-09-16T08:40:51.594 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.794+0000 7f37f4852640 1 -- start start 2024-09-16T08:40:51.594 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.795+0000 7f37f4852640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f37f010aab0 con 0x7f37f0079b30 2024-09-16T08:40:51.594 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.795+0000 7f37ee59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f37f0079b30 0x7f37f0109390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:40144/0 (socket says 172.21.15.160:40144) 2024-09-16T08:40:51.594 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.795+0000 7f37ee59c640 1 -- 172.21.15.160:0/1449445086 learned_addr learned my addr 172.21.15.160:0/1449445086 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:51.595 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.795+0000 7f37ed59a640 1 -- 172.21.15.160:0/1449445086 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4045095949 0 0) 0x7f37f010aab0 con 0x7f37f0079b30 2024-09-16T08:40:51.595 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.795+0000 7f37ed59a640 1 -- 172.21.15.160:0/1449445086 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37d8003520 con 0x7f37f0079b30 2024-09-16T08:40:51.595 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.795+0000 7f37ed59a640 1 -- 172.21.15.160:0/1449445086 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3924700222 0 0) 0x7f37d8003520 con 0x7f37f0079b30 2024-09-16T08:40:51.595 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.795+0000 7f37ed59a640 1 -- 172.21.15.160:0/1449445086 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f010bc90 con 0x7f37f0079b30 2024-09-16T08:40:51.595 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.795+0000 7f37ed59a640 1 -- 172.21.15.160:0/1449445086 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f37e0001600 con 0x7f37f0079b30 2024-09-16T08:40:51.595 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.796+0000 7f37ed59a640 1 -- 172.21.15.160:0/1449445086 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f37e0001d10 con 0x7f37f0079b30 2024-09-16T08:40:51.595 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.796+0000 7f37ed59a640 1 -- 172.21.15.160:0/1449445086 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f37e00058d0 con 0x7f37f0079b30 2024-09-16T08:40:51.595 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.796+0000 7f37f4852640 1 -- 172.21.15.160:0/1449445086 >> v1:172.21.15.160:6789/0 conn(0x7f37f0079b30 legacy=0x7f37f0109390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:51.595 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.796+0000 7f37f4852640 1 -- 172.21.15.160:0/1449445086 shutdown_connections 2024-09-16T08:40:51.595 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.796+0000 7f37f4852640 1 -- 172.21.15.160:0/1449445086 >> 172.21.15.160:0/1449445086 conn(0x7f37f0100f80 msgr2=0x7f37f01033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:51.595 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.796+0000 7f37f4852640 1 -- 172.21.15.160:0/1449445086 shutdown_connections 2024-09-16T08:40:51.595 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.796+0000 7f37f4852640 1 -- 172.21.15.160:0/1449445086 wait complete. 2024-09-16T08:40:51.596 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.797+0000 7f37f4852640 1 Processor -- start 2024-09-16T08:40:51.596 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.797+0000 7f37f4852640 1 -- start start 2024-09-16T08:40:51.596 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.797+0000 7f37f4852640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f37f019aa70 con 0x7f37f0079b30 2024-09-16T08:40:51.596 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.797+0000 7f37ee59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f37f0079b30 0x7f37f019a310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:40146/0 (socket says 172.21.15.160:40146) 2024-09-16T08:40:51.596 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.797+0000 7f37ee59c640 1 -- 172.21.15.160:0/2027700623 learned_addr learned my addr 172.21.15.160:0/2027700623 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:51.596 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.797+0000 7f37df7fe640 1 -- 172.21.15.160:0/2027700623 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 334879454 0 0) 0x7f37f019aa70 con 0x7f37f0079b30 2024-09-16T08:40:51.596 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.797+0000 7f37df7fe640 1 -- 172.21.15.160:0/2027700623 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37d8005570 con 0x7f37f0079b30 2024-09-16T08:40:51.596 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.798+0000 7f37df7fe640 1 -- 172.21.15.160:0/2027700623 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2674428835 0 0) 0x7f37d8005570 con 0x7f37f0079b30 2024-09-16T08:40:51.596 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.798+0000 7f37df7fe640 1 -- 172.21.15.160:0/2027700623 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f37f019aa70 con 0x7f37f0079b30 2024-09-16T08:40:51.596 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.798+0000 7f37df7fe640 1 -- 172.21.15.160:0/2027700623 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f37e0001080 con 0x7f37f0079b30 2024-09-16T08:40:51.596 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.798+0000 7f37df7fe640 1 -- 172.21.15.160:0/2027700623 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 86035349 0 0) 0x7f37f019aa70 con 0x7f37f0079b30 2024-09-16T08:40:51.596 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.798+0000 7f37df7fe640 1 -- 172.21.15.160:0/2027700623 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f019ac60 con 0x7f37f0079b30 2024-09-16T08:40:51.596 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.798+0000 7f37f4852640 1 -- 172.21.15.160:0/2027700623 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f37f019e540 con 0x7f37f0079b30 2024-09-16T08:40:51.596 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.798+0000 7f37f4852640 1 -- 172.21.15.160:0/2027700623 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f37f019e960 con 0x7f37f0079b30 2024-09-16T08:40:51.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.798+0000 7f37df7fe640 1 -- 172.21.15.160:0/2027700623 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f37e00055b0 con 0x7f37f0079b30 2024-09-16T08:40:51.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.798+0000 7f37df7fe640 1 -- 172.21.15.160:0/2027700623 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f37e0006030 con 0x7f37f0079b30 2024-09-16T08:40:51.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.799+0000 7f37f4852640 1 -- 172.21.15.160:0/2027700623 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d00050b0 con 0x7f37f0079b30 2024-09-16T08:40:51.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.799+0000 7f37df7fe640 1 -- 172.21.15.160:0/2027700623 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 12) ==== 51743+0+0 (unknown 4227151905 0 0) 0x7f37e0012ca0 con 0x7f37f0079b30 2024-09-16T08:40:51.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.799+0000 7f37df7fe640 1 -- 172.21.15.160:0/2027700623 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 2761826324 0 0) 0x7f37e004f920 con 0x7f37f0079b30 2024-09-16T08:40:51.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.804+0000 7f37df7fe640 1 -- 172.21.15.160:0/2027700623 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f37e0018f70 con 0x7f37f0079b30 2024-09-16T08:40:51.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.993+0000 7f37f4852640 1 -- 172.21.15.160:0/2027700623 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}) -- 0x7f37d0002420 con 0x7f37d80408b0 2024-09-16T08:40:51.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.997+0000 7f37df7fe640 1 -- 172.21.15.160:0/2027700623 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (unknown 0 0 3881938646) 0x7f37d0002420 con 0x7f37d80408b0 2024-09-16T08:40:51.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.999+0000 7f37f4852640 1 -- 172.21.15.160:0/2027700623 >> v1:172.21.15.160:6800/1286008384 conn(0x7f37d80408b0 legacy=0x7f37d8042d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:51.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.999+0000 7f37f4852640 1 -- 172.21.15.160:0/2027700623 >> v1:172.21.15.160:6789/0 conn(0x7f37f0079b30 legacy=0x7f37f019a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:51.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.999+0000 7f37f4852640 1 -- 172.21.15.160:0/2027700623 shutdown_connections 2024-09-16T08:40:51.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.999+0000 7f37f4852640 1 -- 172.21.15.160:0/2027700623 >> 172.21.15.160:0/2027700623 conn(0x7f37f0100f80 msgr2=0x7f37f0191060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:51.597 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.999+0000 7f37f4852640 1 -- 172.21.15.160:0/2027700623 shutdown_connections 2024-09-16T08:40:51.598 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:50.999+0000 7f37f4852640 1 -- 172.21.15.160:0/2027700623 wait complete. 2024-09-16T08:40:52.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:51 smithi160 ceph-mon[30137]: from='client.14164 v1:172.21.15.160:0/2027700623' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:40:52.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:51 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:40:52.851 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-16T08:40:52.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.043+0000 7fd75a328640 1 Processor -- start 2024-09-16T08:40:52.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.043+0000 7fd75a328640 1 -- start start 2024-09-16T08:40:52.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.043+0000 7fd75a328640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd754109ad0 con 0x7fd754105650 2024-09-16T08:40:52.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.043+0000 7fd753fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd754105650 0x7fd754105a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:40148/0 (socket says 172.21.15.160:40148) 2024-09-16T08:40:52.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.044+0000 7fd753fff640 1 -- 172.21.15.160:0/2889490201 learned_addr learned my addr 172.21.15.160:0/2889490201 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:52.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.044+0000 7fd752ffd640 1 -- 172.21.15.160:0/2889490201 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3480531039 0 0) 0x7fd754109ad0 con 0x7fd754105650 2024-09-16T08:40:52.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.044+0000 7fd752ffd640 1 -- 172.21.15.160:0/2889490201 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd73c005730 con 0x7fd754105650 2024-09-16T08:40:52.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.044+0000 7fd752ffd640 1 -- 172.21.15.160:0/2889490201 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1263226004 0 0) 0x7fd73c005730 con 0x7fd754105650 2024-09-16T08:40:52.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.044+0000 7fd752ffd640 1 -- 172.21.15.160:0/2889490201 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75410acb0 con 0x7fd754105650 2024-09-16T08:40:52.852 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.044+0000 7fd752ffd640 1 -- 172.21.15.160:0/2889490201 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fd744001600 con 0x7fd754105650 2024-09-16T08:40:52.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.044+0000 7fd752ffd640 1 -- 172.21.15.160:0/2889490201 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fd744001d10 con 0x7fd754105650 2024-09-16T08:40:52.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.044+0000 7fd752ffd640 1 -- 172.21.15.160:0/2889490201 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fd7440058d0 con 0x7fd754105650 2024-09-16T08:40:52.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.045+0000 7fd75a328640 1 -- 172.21.15.160:0/2889490201 >> v1:172.21.15.160:6789/0 conn(0x7fd754105650 legacy=0x7fd754105a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:52.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.045+0000 7fd75a328640 1 -- 172.21.15.160:0/2889490201 shutdown_connections 2024-09-16T08:40:52.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.045+0000 7fd75a328640 1 -- 172.21.15.160:0/2889490201 >> 172.21.15.160:0/2889490201 conn(0x7fd754100f80 msgr2=0x7fd7541033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:52.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.045+0000 7fd75a328640 1 -- 172.21.15.160:0/2889490201 shutdown_connections 2024-09-16T08:40:52.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.045+0000 7fd75a328640 1 -- 172.21.15.160:0/2889490201 wait complete. 2024-09-16T08:40:52.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.045+0000 7fd75a328640 1 Processor -- start 2024-09-16T08:40:52.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.045+0000 7fd75a328640 1 -- start start 2024-09-16T08:40:52.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.046+0000 7fd75a328640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd754198890 con 0x7fd754105650 2024-09-16T08:40:52.853 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.046+0000 7fd753fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd754105650 0x7fd754198130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:40154/0 (socket says 172.21.15.160:40154) 2024-09-16T08:40:52.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.046+0000 7fd753fff640 1 -- 172.21.15.160:0/2548919002 learned_addr learned my addr 172.21.15.160:0/2548919002 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:52.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.046+0000 7fd7517fa640 1 -- 172.21.15.160:0/2548919002 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2444519133 0 0) 0x7fd754198890 con 0x7fd754105650 2024-09-16T08:40:52.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.046+0000 7fd7517fa640 1 -- 172.21.15.160:0/2548919002 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd73c007780 con 0x7fd754105650 2024-09-16T08:40:52.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.046+0000 7fd7517fa640 1 -- 172.21.15.160:0/2548919002 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1665209072 0 0) 0x7fd73c007780 con 0x7fd754105650 2024-09-16T08:40:52.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.046+0000 7fd7517fa640 1 -- 172.21.15.160:0/2548919002 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd754198890 con 0x7fd754105650 2024-09-16T08:40:52.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.046+0000 7fd7517fa640 1 -- 172.21.15.160:0/2548919002 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fd744001080 con 0x7fd754105650 2024-09-16T08:40:52.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.046+0000 7fd7517fa640 1 -- 172.21.15.160:0/2548919002 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1983769782 0 0) 0x7fd754198890 con 0x7fd754105650 2024-09-16T08:40:52.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.046+0000 7fd7517fa640 1 -- 172.21.15.160:0/2548919002 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd754198a80 con 0x7fd754105650 2024-09-16T08:40:52.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.046+0000 7fd75a328640 1 -- 172.21.15.160:0/2548919002 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd75419c360 con 0x7fd754105650 2024-09-16T08:40:52.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.046+0000 7fd75a328640 1 -- 172.21.15.160:0/2548919002 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd75419c780 con 0x7fd754105650 2024-09-16T08:40:52.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.046+0000 7fd7517fa640 1 -- 172.21.15.160:0/2548919002 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fd744001ad0 con 0x7fd754105650 2024-09-16T08:40:52.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.046+0000 7fd7517fa640 1 -- 172.21.15.160:0/2548919002 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fd744006390 con 0x7fd754105650 2024-09-16T08:40:52.854 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.047+0000 7fd7517fa640 1 -- 172.21.15.160:0/2548919002 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 12) ==== 51743+0+0 (unknown 4227151905 0 0) 0x7fd7440065d0 con 0x7fd754105650 2024-09-16T08:40:52.855 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.047+0000 7fd75a328640 1 -- 172.21.15.160:0/2548919002 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7400050b0 con 0x7fd754105650 2024-09-16T08:40:52.855 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.047+0000 7fd7517fa640 1 -- 172.21.15.160:0/2548919002 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 2761826324 0 0) 0x7fd74404fa10 con 0x7fd754105650 2024-09-16T08:40:52.855 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.052+0000 7fd7517fa640 1 -- 172.21.15.160:0/2548919002 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fd744018f70 con 0x7fd754105650 2024-09-16T08:40:52.855 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.242+0000 7fd75a328640 1 -- 172.21.15.160:0/2548919002 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}) -- 0x7fd7400053a0 con 0x7fd73c042ab0 2024-09-16T08:40:52.855 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.246+0000 7fd7517fa640 1 -- 172.21.15.160:0/2548919002 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (unknown 0 0 3070210822) 0x7fd7400053a0 con 0x7fd73c042ab0 2024-09-16T08:40:52.855 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.248+0000 7fd75a328640 1 -- 172.21.15.160:0/2548919002 >> v1:172.21.15.160:6800/1286008384 conn(0x7fd73c042ab0 legacy=0x7fd73c044f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:52.855 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.248+0000 7fd75a328640 1 -- 172.21.15.160:0/2548919002 >> v1:172.21.15.160:6789/0 conn(0x7fd754105650 legacy=0x7fd754198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:52.855 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.248+0000 7fd75a328640 1 -- 172.21.15.160:0/2548919002 shutdown_connections 2024-09-16T08:40:52.855 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.248+0000 7fd75a328640 1 -- 172.21.15.160:0/2548919002 >> 172.21.15.160:0/2548919002 conn(0x7fd754100f80 msgr2=0x7fd754103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:52.855 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.248+0000 7fd75a328640 1 -- 172.21.15.160:0/2548919002 shutdown_connections 2024-09-16T08:40:52.855 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:52.248+0000 7fd75a328640 1 -- 172.21.15.160:0/2548919002 wait complete. 2024-09-16T08:40:52.855 INFO:teuthology.orchestra.run.smithi160.stdout:Creating initial admin user... 2024-09-16T08:40:53.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:53 smithi160 ceph-mon[30137]: from='client.14166 v1:172.21.15.160:0/2548919002' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:40:53.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:53 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:40:54.265 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$ViFVQ7EHsoHmlrKdibnecOTEK2YopggZmQ4S/mdbXdLqwDjOsStd2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726476053, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-16T08:40:54.265 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.321+0000 7fe5c27a7640 1 Processor -- start 2024-09-16T08:40:54.265 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.321+0000 7fe5c27a7640 1 -- start start 2024-09-16T08:40:54.266 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.321+0000 7fe5c27a7640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe5bc109ac0 con 0x7fe5bc105640 2024-09-16T08:40:54.266 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.321+0000 7fe5bbfff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fe5bc105640 0x7fe5bc105a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:40162/0 (socket says 172.21.15.160:40162) 2024-09-16T08:40:54.266 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.321+0000 7fe5bbfff640 1 -- 172.21.15.160:0/3616363496 learned_addr learned my addr 172.21.15.160:0/3616363496 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:54.266 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.321+0000 7fe5baffd640 1 -- 172.21.15.160:0/3616363496 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 703304522 0 0) 0x7fe5bc109ac0 con 0x7fe5bc105640 2024-09-16T08:40:54.266 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.321+0000 7fe5baffd640 1 -- 172.21.15.160:0/3616363496 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5a4003520 con 0x7fe5bc105640 2024-09-16T08:40:54.266 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.322+0000 7fe5baffd640 1 -- 172.21.15.160:0/3616363496 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4067250091 0 0) 0x7fe5a4003520 con 0x7fe5bc105640 2024-09-16T08:40:54.266 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.322+0000 7fe5baffd640 1 -- 172.21.15.160:0/3616363496 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5bc10aca0 con 0x7fe5bc105640 2024-09-16T08:40:54.266 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.322+0000 7fe5baffd640 1 -- 172.21.15.160:0/3616363496 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fe5ac001600 con 0x7fe5bc105640 2024-09-16T08:40:54.266 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.322+0000 7fe5baffd640 1 -- 172.21.15.160:0/3616363496 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fe5ac001d10 con 0x7fe5bc105640 2024-09-16T08:40:54.266 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.322+0000 7fe5baffd640 1 -- 172.21.15.160:0/3616363496 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fe5ac0058d0 con 0x7fe5bc105640 2024-09-16T08:40:54.266 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.322+0000 7fe5c27a7640 1 -- 172.21.15.160:0/3616363496 >> v1:172.21.15.160:6789/0 conn(0x7fe5bc105640 legacy=0x7fe5bc105a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:54.266 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.322+0000 7fe5c27a7640 1 -- 172.21.15.160:0/3616363496 shutdown_connections 2024-09-16T08:40:54.267 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.322+0000 7fe5c27a7640 1 -- 172.21.15.160:0/3616363496 >> 172.21.15.160:0/3616363496 conn(0x7fe5bc100f30 msgr2=0x7fe5bc103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:54.267 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.322+0000 7fe5c27a7640 1 -- 172.21.15.160:0/3616363496 shutdown_connections 2024-09-16T08:40:54.267 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.322+0000 7fe5c27a7640 1 -- 172.21.15.160:0/3616363496 wait complete. 2024-09-16T08:40:54.267 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.323+0000 7fe5c27a7640 1 Processor -- start 2024-09-16T08:40:54.267 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.323+0000 7fe5c27a7640 1 -- start start 2024-09-16T08:40:54.267 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.323+0000 7fe5c27a7640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe5bc19aa80 con 0x7fe5bc105640 2024-09-16T08:40:54.267 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.323+0000 7fe5bbfff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fe5bc105640 0x7fe5bc19a320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:40166/0 (socket says 172.21.15.160:40166) 2024-09-16T08:40:54.267 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.323+0000 7fe5bbfff640 1 -- 172.21.15.160:0/505425144 learned_addr learned my addr 172.21.15.160:0/505425144 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:54.267 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.323+0000 7fe5b97fa640 1 -- 172.21.15.160:0/505425144 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3082861664 0 0) 0x7fe5bc19aa80 con 0x7fe5bc105640 2024-09-16T08:40:54.267 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.324+0000 7fe5b97fa640 1 -- 172.21.15.160:0/505425144 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5a4005570 con 0x7fe5bc105640 2024-09-16T08:40:54.267 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.324+0000 7fe5b97fa640 1 -- 172.21.15.160:0/505425144 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1734760358 0 0) 0x7fe5a4005570 con 0x7fe5bc105640 2024-09-16T08:40:54.267 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.324+0000 7fe5b97fa640 1 -- 172.21.15.160:0/505425144 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe5bc19aa80 con 0x7fe5bc105640 2024-09-16T08:40:54.267 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.324+0000 7fe5b97fa640 1 -- 172.21.15.160:0/505425144 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fe5ac001080 con 0x7fe5bc105640 2024-09-16T08:40:54.268 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.324+0000 7fe5b97fa640 1 -- 172.21.15.160:0/505425144 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 645291855 0 0) 0x7fe5bc19aa80 con 0x7fe5bc105640 2024-09-16T08:40:54.268 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.324+0000 7fe5b97fa640 1 -- 172.21.15.160:0/505425144 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5bc19ac70 con 0x7fe5bc105640 2024-09-16T08:40:54.268 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.324+0000 7fe5c27a7640 1 -- 172.21.15.160:0/505425144 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe5bc19e550 con 0x7fe5bc105640 2024-09-16T08:40:54.268 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.324+0000 7fe5c27a7640 1 -- 172.21.15.160:0/505425144 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe5bc19e970 con 0x7fe5bc105640 2024-09-16T08:40:54.268 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.324+0000 7fe5b97fa640 1 -- 172.21.15.160:0/505425144 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fe5ac0055b0 con 0x7fe5bc105640 2024-09-16T08:40:54.268 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.324+0000 7fe5b97fa640 1 -- 172.21.15.160:0/505425144 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fe5ac006030 con 0x7fe5bc105640 2024-09-16T08:40:54.268 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.325+0000 7fe5b97fa640 1 -- 172.21.15.160:0/505425144 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 12) ==== 51743+0+0 (unknown 4227151905 0 0) 0x7fe5ac012ca0 con 0x7fe5bc105640 2024-09-16T08:40:54.268 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.325+0000 7fe5c27a7640 1 -- 172.21.15.160:0/505425144 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a80050b0 con 0x7fe5bc105640 2024-09-16T08:40:54.268 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.325+0000 7fe5b97fa640 1 -- 172.21.15.160:0/505425144 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 2761826324 0 0) 0x7fe5ac04fa60 con 0x7fe5bc105640 2024-09-16T08:40:54.268 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.330+0000 7fe5b97fa640 1 -- 172.21.15.160:0/505425144 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fe5ac018f70 con 0x7fe5bc105640 2024-09-16T08:40:54.268 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.522+0000 7fe5c27a7640 1 -- 172.21.15.160:0/505425144 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fe5a8002420 con 0x7fe5a40408b0 2024-09-16T08:40:54.268 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.741+0000 7fe5b97fa640 1 -- 172.21.15.160:0/505425144 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (unknown 0 0 3479804608) 0x7fe5a8002420 con 0x7fe5a40408b0 2024-09-16T08:40:54.269 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.744+0000 7fe5c27a7640 1 -- 172.21.15.160:0/505425144 >> v1:172.21.15.160:6800/1286008384 conn(0x7fe5a40408b0 legacy=0x7fe5a4042d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:54.269 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.744+0000 7fe5c27a7640 1 -- 172.21.15.160:0/505425144 >> v1:172.21.15.160:6789/0 conn(0x7fe5bc105640 legacy=0x7fe5bc19a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:54.269 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.744+0000 7fe5c27a7640 1 -- 172.21.15.160:0/505425144 shutdown_connections 2024-09-16T08:40:54.269 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.744+0000 7fe5c27a7640 1 -- 172.21.15.160:0/505425144 >> 172.21.15.160:0/505425144 conn(0x7fe5bc100f30 msgr2=0x7fe5bc109450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:54.269 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.744+0000 7fe5c27a7640 1 -- 172.21.15.160:0/505425144 shutdown_connections 2024-09-16T08:40:54.269 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:53.744+0000 7fe5c27a7640 1 -- 172.21.15.160:0/505425144 wait complete. 2024-09-16T08:40:54.269 INFO:teuthology.orchestra.run.smithi160.stdout:Fetching dashboard port number... 2024-09-16T08:40:54.902 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:54 smithi160 ceph-mon[30137]: from='client.14168 v1:172.21.15.160:0/505425144' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:40:54.902 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:54 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:40:55.487 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 8443 2024-09-16T08:40:55.487 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.715+0000 7fa059d0b640 1 Processor -- start 2024-09-16T08:40:55.487 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.715+0000 7fa059d0b640 1 -- start start 2024-09-16T08:40:55.487 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.716+0000 7fa059d0b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa054109aa0 con 0x7fa054105620 2024-09-16T08:40:55.487 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.716+0000 7fa0537fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fa054105620 0x7fa054105a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:40178/0 (socket says 172.21.15.160:40178) 2024-09-16T08:40:55.488 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.716+0000 7fa0537fe640 1 -- 172.21.15.160:0/1748497626 learned_addr learned my addr 172.21.15.160:0/1748497626 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:55.488 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.716+0000 7fa0527fc640 1 -- 172.21.15.160:0/1748497626 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3560011226 0 0) 0x7fa054109aa0 con 0x7fa054105620 2024-09-16T08:40:55.488 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.716+0000 7fa0527fc640 1 -- 172.21.15.160:0/1748497626 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa03c003520 con 0x7fa054105620 2024-09-16T08:40:55.488 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.716+0000 7fa0527fc640 1 -- 172.21.15.160:0/1748497626 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 322217625 0 0) 0x7fa03c003520 con 0x7fa054105620 2024-09-16T08:40:55.488 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.716+0000 7fa0527fc640 1 -- 172.21.15.160:0/1748497626 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05410ac80 con 0x7fa054105620 2024-09-16T08:40:55.488 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.716+0000 7fa0527fc640 1 -- 172.21.15.160:0/1748497626 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fa044001600 con 0x7fa054105620 2024-09-16T08:40:55.488 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.716+0000 7fa0527fc640 1 -- 172.21.15.160:0/1748497626 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fa044001d10 con 0x7fa054105620 2024-09-16T08:40:55.488 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.716+0000 7fa0527fc640 1 -- 172.21.15.160:0/1748497626 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fa0440058d0 con 0x7fa054105620 2024-09-16T08:40:55.488 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.717+0000 7fa059d0b640 1 -- 172.21.15.160:0/1748497626 >> v1:172.21.15.160:6789/0 conn(0x7fa054105620 legacy=0x7fa054105a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:55.488 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.717+0000 7fa059d0b640 1 -- 172.21.15.160:0/1748497626 shutdown_connections 2024-09-16T08:40:55.488 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.717+0000 7fa059d0b640 1 -- 172.21.15.160:0/1748497626 >> 172.21.15.160:0/1748497626 conn(0x7fa054100f90 msgr2=0x7fa0541033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:55.488 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.717+0000 7fa059d0b640 1 -- 172.21.15.160:0/1748497626 shutdown_connections 2024-09-16T08:40:55.488 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.717+0000 7fa059d0b640 1 -- 172.21.15.160:0/1748497626 wait complete. 2024-09-16T08:40:55.489 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.717+0000 7fa059d0b640 1 Processor -- start 2024-09-16T08:40:55.489 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.718+0000 7fa059d0b640 1 -- start start 2024-09-16T08:40:55.489 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.718+0000 7fa059d0b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa05419e000 con 0x7fa054105620 2024-09-16T08:40:55.489 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.718+0000 7fa0537fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fa054105620 0x7fa05407a050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:40186/0 (socket says 172.21.15.160:40186) 2024-09-16T08:40:55.489 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.718+0000 7fa0537fe640 1 -- 172.21.15.160:0/3687016553 learned_addr learned my addr 172.21.15.160:0/3687016553 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:55.489 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.718+0000 7fa050ff9640 1 -- 172.21.15.160:0/3687016553 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1904850626 0 0) 0x7fa05419e000 con 0x7fa054105620 2024-09-16T08:40:55.489 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.718+0000 7fa050ff9640 1 -- 172.21.15.160:0/3687016553 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa03c005570 con 0x7fa054105620 2024-09-16T08:40:55.489 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.718+0000 7fa050ff9640 1 -- 172.21.15.160:0/3687016553 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1908536911 0 0) 0x7fa03c005570 con 0x7fa054105620 2024-09-16T08:40:55.489 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.718+0000 7fa050ff9640 1 -- 172.21.15.160:0/3687016553 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa05419e000 con 0x7fa054105620 2024-09-16T08:40:55.489 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.718+0000 7fa050ff9640 1 -- 172.21.15.160:0/3687016553 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fa044001080 con 0x7fa054105620 2024-09-16T08:40:55.489 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.718+0000 7fa050ff9640 1 -- 172.21.15.160:0/3687016553 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 181729795 0 0) 0x7fa05419e000 con 0x7fa054105620 2024-09-16T08:40:55.489 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.718+0000 7fa050ff9640 1 -- 172.21.15.160:0/3687016553 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05419f1e0 con 0x7fa054105620 2024-09-16T08:40:55.489 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.718+0000 7fa059d0b640 1 -- 172.21.15.160:0/3687016553 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa05419e1d0 con 0x7fa054105620 2024-09-16T08:40:55.489 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.718+0000 7fa059d0b640 1 -- 172.21.15.160:0/3687016553 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa05419e760 con 0x7fa054105620 2024-09-16T08:40:55.490 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.718+0000 7fa050ff9640 1 -- 172.21.15.160:0/3687016553 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fa0440055b0 con 0x7fa054105620 2024-09-16T08:40:55.490 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.718+0000 7fa050ff9640 1 -- 172.21.15.160:0/3687016553 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fa044006030 con 0x7fa054105620 2024-09-16T08:40:55.490 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.719+0000 7fa059d0b640 1 -- 172.21.15.160:0/3687016553 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0400050b0 con 0x7fa054105620 2024-09-16T08:40:55.490 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.720+0000 7fa050ff9640 1 -- 172.21.15.160:0/3687016553 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 12) ==== 51743+0+0 (unknown 4227151905 0 0) 0x7fa044012c20 con 0x7fa054105620 2024-09-16T08:40:55.490 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.720+0000 7fa050ff9640 1 -- 172.21.15.160:0/3687016553 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 2761826324 0 0) 0x7fa04404f7f0 con 0x7fa054105620 2024-09-16T08:40:55.490 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.724+0000 7fa050ff9640 1 -- 172.21.15.160:0/3687016553 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fa044018ea0 con 0x7fa054105620 2024-09-16T08:40:55.490 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.900+0000 7fa059d0b640 1 -- 172.21.15.160:0/3687016553 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fa0400053a0 con 0x7fa054105620 2024-09-16T08:40:55.490 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.901+0000 7fa050ff9640 1 -- 172.21.15.160:0/3687016553 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (unknown 3713421687 0 83753974) 0x7fa0440187a0 con 0x7fa054105620 2024-09-16T08:40:55.490 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.903+0000 7fa059d0b640 1 -- 172.21.15.160:0/3687016553 >> v1:172.21.15.160:6800/1286008384 conn(0x7fa03c040900 legacy=0x7fa03c042dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:55.490 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.903+0000 7fa059d0b640 1 -- 172.21.15.160:0/3687016553 >> v1:172.21.15.160:6789/0 conn(0x7fa054105620 legacy=0x7fa05407a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:55.490 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.903+0000 7fa059d0b640 1 -- 172.21.15.160:0/3687016553 shutdown_connections 2024-09-16T08:40:55.490 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.903+0000 7fa059d0b640 1 -- 172.21.15.160:0/3687016553 >> 172.21.15.160:0/3687016553 conn(0x7fa054100f90 msgr2=0x7fa0541033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:55.490 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.903+0000 7fa059d0b640 1 -- 172.21.15.160:0/3687016553 shutdown_connections 2024-09-16T08:40:55.491 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:54.903+0000 7fa059d0b640 1 -- 172.21.15.160:0/3687016553 wait complete. 2024-09-16T08:40:55.495 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T08:40:55.495 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout disabled 2024-09-16T08:40:55.502 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T08:40:55.502 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-09-16T08:40:55.502 INFO:teuthology.orchestra.run.smithi160.stdout:firewalld.service is not enabled 2024-09-16T08:40:55.502 INFO:teuthology.orchestra.run.smithi160.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-16T08:40:55.503 INFO:teuthology.orchestra.run.smithi160.stdout:Ceph Dashboard is now available at: 2024-09-16T08:40:55.503 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:40:55.503 INFO:teuthology.orchestra.run.smithi160.stdout: URL: https://smithi160.front.sepia.ceph.com:8443/ 2024-09-16T08:40:55.503 INFO:teuthology.orchestra.run.smithi160.stdout: User: admin 2024-09-16T08:40:55.503 INFO:teuthology.orchestra.run.smithi160.stdout: Password: dpttnsbe7z 2024-09-16T08:40:55.504 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:40:55.504 INFO:teuthology.orchestra.run.smithi160.stdout:Saving cluster configuration to /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config directory 2024-09-16T08:40:56.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3687016553' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-16T08:40:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.954+0000 7f7b72aaa640 1 Processor -- start 2024-09-16T08:40:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.954+0000 7f7b72aaa640 1 -- start start 2024-09-16T08:40:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.955+0000 7f7b72aaa640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7b6c07b230 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.955+0000 7f7b70846640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f7b6c07c4c0 0x7f7b6c07ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:40190/0 (socket says 172.21.15.160:40190) 2024-09-16T08:40:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.955+0000 7f7b70846640 1 -- 172.21.15.160:0/2532076231 learned_addr learned my addr 172.21.15.160:0/2532076231 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.955+0000 7f7b6b7fe640 1 -- 172.21.15.160:0/2532076231 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2787806935 0 0) 0x7f7b6c07b230 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.955+0000 7f7b6b7fe640 1 -- 172.21.15.160:0/2532076231 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b5c003520 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.836 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.955+0000 7f7b6b7fe640 1 -- 172.21.15.160:0/2532076231 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4111000245 0 0) 0x7f7b5c003520 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.955+0000 7f7b6b7fe640 1 -- 172.21.15.160:0/2532076231 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c07b400 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.955+0000 7f7b6b7fe640 1 -- 172.21.15.160:0/2532076231 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f7b60001600 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.955+0000 7f7b6b7fe640 1 -- 172.21.15.160:0/2532076231 <== mon.0 v1:172.21.15.160:6789/0 4 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f7b60001d10 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.955+0000 7f7b6b7fe640 1 -- 172.21.15.160:0/2532076231 <== mon.0 v1:172.21.15.160:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f7b600058d0 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.956+0000 7f7b72aaa640 1 -- 172.21.15.160:0/2532076231 >> v1:172.21.15.160:6789/0 conn(0x7f7b6c07c4c0 legacy=0x7f7b6c07ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.956+0000 7f7b72aaa640 1 -- 172.21.15.160:0/2532076231 shutdown_connections 2024-09-16T08:40:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.956+0000 7f7b72aaa640 1 -- 172.21.15.160:0/2532076231 >> 172.21.15.160:0/2532076231 conn(0x7f7b6c102070 msgr2=0x7f7b6c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.956+0000 7f7b72aaa640 1 -- 172.21.15.160:0/2532076231 shutdown_connections 2024-09-16T08:40:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.956+0000 7f7b72aaa640 1 -- 172.21.15.160:0/2532076231 wait complete. 2024-09-16T08:40:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.956+0000 7f7b72aaa640 1 Processor -- start 2024-09-16T08:40:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.956+0000 7f7b72aaa640 1 -- start start 2024-09-16T08:40:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.957+0000 7f7b72aaa640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7b6c19cc30 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.957+0000 7f7b70846640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f7b6c07c4c0 0x7f7b6c19c4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:40196/0 (socket says 172.21.15.160:40196) 2024-09-16T08:40:56.837 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.957+0000 7f7b70846640 1 -- 172.21.15.160:0/1106768145 learned_addr learned my addr 172.21.15.160:0/1106768145 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:56.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.957+0000 7f7b69ffb640 1 -- 172.21.15.160:0/1106768145 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1653524246 0 0) 0x7f7b6c19cc30 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.957+0000 7f7b69ffb640 1 -- 172.21.15.160:0/1106768145 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b5c005570 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.957+0000 7f7b69ffb640 1 -- 172.21.15.160:0/1106768145 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1613087448 0 0) 0x7f7b5c005570 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.957+0000 7f7b69ffb640 1 -- 172.21.15.160:0/1106768145 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7b6c19cc30 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.957+0000 7f7b69ffb640 1 -- 172.21.15.160:0/1106768145 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f7b60001080 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.957+0000 7f7b69ffb640 1 -- 172.21.15.160:0/1106768145 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2529104784 0 0) 0x7f7b6c19cc30 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.957+0000 7f7b69ffb640 1 -- 172.21.15.160:0/1106768145 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c19ce20 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.957+0000 7f7b72aaa640 1 -- 172.21.15.160:0/1106768145 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c1a0700 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.957+0000 7f7b69ffb640 1 -- 172.21.15.160:0/1106768145 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f7b600055b0 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.958+0000 7f7b69ffb640 1 -- 172.21.15.160:0/1106768145 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f7b60006380 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.958+0000 7f7b72aaa640 1 -- 172.21.15.160:0/1106768145 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7b6c1a0b20 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.959+0000 7f7b69ffb640 1 -- 172.21.15.160:0/1106768145 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 12) ==== 51743+0+0 (unknown 4227151905 0 0) 0x7f7b60012f70 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.959+0000 7f7b72aaa640 1 -- 172.21.15.160:0/1106768145 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b6c1958f0 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.838 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.959+0000 7f7b69ffb640 1 -- 172.21.15.160:0/1106768145 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 2761826324 0 0) 0x7f7b6004f890 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:55.963+0000 7f7b69ffb640 1 -- 172.21.15.160:0/1106768145 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f7b60018e70 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:56.237+0000 7f7b72aaa640 1 -- 172.21.15.160:0/1106768145 --> v1:172.21.15.160:6789/0 -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f7b6c10de00 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:56.240+0000 7f7b69ffb640 1 -- 172.21.15.160:0/1106768145 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (unknown 1169358022 0 0) 0x7f7b60018770 con 0x7f7b6c07c4c0 2024-09-16T08:40:56.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-16T08:40:56.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:56.242+0000 7f7b72aaa640 1 -- 172.21.15.160:0/1106768145 >> v1:172.21.15.160:6800/1286008384 conn(0x7f7b5c040900 legacy=0x7f7b5c042dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:56.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:56.242+0000 7f7b72aaa640 1 -- 172.21.15.160:0/1106768145 >> v1:172.21.15.160:6789/0 conn(0x7f7b6c07c4c0 legacy=0x7f7b6c19c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:56.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:56.242+0000 7f7b72aaa640 1 -- 172.21.15.160:0/1106768145 shutdown_connections 2024-09-16T08:40:56.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:56.242+0000 7f7b72aaa640 1 -- 172.21.15.160:0/1106768145 >> 172.21.15.160:0/1106768145 conn(0x7f7b6c102070 msgr2=0x7f7b6c10cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:56.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:56.242+0000 7f7b72aaa640 1 -- 172.21.15.160:0/1106768145 shutdown_connections 2024-09-16T08:40:56.839 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-09-16T08:40:56.242+0000 7f7b72aaa640 1 -- 172.21.15.160:0/1106768145 wait complete. 2024-09-16T08:40:56.839 INFO:teuthology.orchestra.run.smithi160.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-16T08:40:56.839 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:40:56.839 INFO:teuthology.orchestra.run.smithi160.stdout: sudo /sbin/cephadm shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-16T08:40:56.839 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:40:56.839 INFO:teuthology.orchestra.run.smithi160.stdout:Or, if you are only running a single cluster on this host: 2024-09-16T08:40:56.840 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:40:56.840 INFO:teuthology.orchestra.run.smithi160.stdout: sudo /sbin/cephadm shell 2024-09-16T08:40:56.840 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:40:56.840 INFO:teuthology.orchestra.run.smithi160.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-16T08:40:56.840 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:40:56.840 INFO:teuthology.orchestra.run.smithi160.stdout: ceph telemetry on 2024-09-16T08:40:56.840 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:40:56.840 INFO:teuthology.orchestra.run.smithi160.stdout:For more information see: 2024-09-16T08:40:56.840 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:40:56.840 INFO:teuthology.orchestra.run.smithi160.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-16T08:40:56.840 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:40:56.840 INFO:teuthology.orchestra.run.smithi160.stdout:Bootstrap complete. 2024-09-16T08:40:56.873 INFO:tasks.cephadm:Fetching config... 2024-09-16T08:40:56.874 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:40:56.874 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-16T08:40:56.890 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-16T08:40:56.891 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:40:56.891 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-16T08:40:56.948 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-16T08:40:56.948 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:40:56.948 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-16T08:40:57.013 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-16T08:40:57.013 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:40:57.013 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-16T08:40:57.068 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-16T08:40:57.069 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFo1RwGOUA+EXBp0a7t2lR/doCyOZPDvvAke0ImskAGdRd1Cok3ia1ioVV+lFtFFpS+n+nQ5ot1KbAMGCeMKQHcUTlrRjDzhUqRIj7XUo7iarzcIfiC2bEgitS0oA0EG5uy8JeiWgfSre+KtFjw7BewLwzKfD3XavrMAgF49dKmqctfxVV3maghdfJCjLP4//CL1NIe8BR8LdLmT9SVwSSgLCDnn+1lrZ1VsD6jg93naVEoQnO0Wfss4T6Tef1YCo48sV0/OY3Hqh1kdDilGjOrMxzwnwkzddJqnZLSaDsQYX+lSKFCgPpNnJMiVDJxw+tHT0RwptIU/Q/7mYYIjwJERlUJUfgNbfAsUGJw9bUg8CGOU0j2Bs/hoRt81xyDxV+7pkjMn1owK1ygXeZAGf97nzcf9xo/6AVt+IFQQHA2oh+4v6Dz7tsuXbktO0Lik6RgRUQHLfe0ThW6ganch7lFOtQBGclNTmPc9A3/nTKGYdXQzLYFnP6sMFnh80RU5M= ceph-27cca2fc-7407-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T08:40:57.157 INFO:teuthology.orchestra.run.smithi160.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFo1RwGOUA+EXBp0a7t2lR/doCyOZPDvvAke0ImskAGdRd1Cok3ia1ioVV+lFtFFpS+n+nQ5ot1KbAMGCeMKQHcUTlrRjDzhUqRIj7XUo7iarzcIfiC2bEgitS0oA0EG5uy8JeiWgfSre+KtFjw7BewLwzKfD3XavrMAgF49dKmqctfxVV3maghdfJCjLP4//CL1NIe8BR8LdLmT9SVwSSgLCDnn+1lrZ1VsD6jg93naVEoQnO0Wfss4T6Tef1YCo48sV0/OY3Hqh1kdDilGjOrMxzwnwkzddJqnZLSaDsQYX+lSKFCgPpNnJMiVDJxw+tHT0RwptIU/Q/7mYYIjwJERlUJUfgNbfAsUGJw9bUg8CGOU0j2Bs/hoRt81xyDxV+7pkjMn1owK1ygXeZAGf97nzcf9xo/6AVt+IFQQHA2oh+4v6Dz7tsuXbktO0Lik6RgRUQHLfe0ThW6ganch7lFOtQBGclNTmPc9A3/nTKGYdXQzLYFnP6sMFnh80RU5M= ceph-27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:40:57.169 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFo1RwGOUA+EXBp0a7t2lR/doCyOZPDvvAke0ImskAGdRd1Cok3ia1ioVV+lFtFFpS+n+nQ5ot1KbAMGCeMKQHcUTlrRjDzhUqRIj7XUo7iarzcIfiC2bEgitS0oA0EG5uy8JeiWgfSre+KtFjw7BewLwzKfD3XavrMAgF49dKmqctfxVV3maghdfJCjLP4//CL1NIe8BR8LdLmT9SVwSSgLCDnn+1lrZ1VsD6jg93naVEoQnO0Wfss4T6Tef1YCo48sV0/OY3Hqh1kdDilGjOrMxzwnwkzddJqnZLSaDsQYX+lSKFCgPpNnJMiVDJxw+tHT0RwptIU/Q/7mYYIjwJERlUJUfgNbfAsUGJw9bUg8CGOU0j2Bs/hoRt81xyDxV+7pkjMn1owK1ygXeZAGf97nzcf9xo/6AVt+IFQQHA2oh+4v6Dz7tsuXbktO0Lik6RgRUQHLfe0ThW6ganch7lFOtQBGclNTmPc9A3/nTKGYdXQzLYFnP6sMFnh80RU5M= ceph-27cca2fc-7407-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T08:40:57.308 INFO:teuthology.orchestra.run.smithi169.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFo1RwGOUA+EXBp0a7t2lR/doCyOZPDvvAke0ImskAGdRd1Cok3ia1ioVV+lFtFFpS+n+nQ5ot1KbAMGCeMKQHcUTlrRjDzhUqRIj7XUo7iarzcIfiC2bEgitS0oA0EG5uy8JeiWgfSre+KtFjw7BewLwzKfD3XavrMAgF49dKmqctfxVV3maghdfJCjLP4//CL1NIe8BR8LdLmT9SVwSSgLCDnn+1lrZ1VsD6jg93naVEoQnO0Wfss4T6Tef1YCo48sV0/OY3Hqh1kdDilGjOrMxzwnwkzddJqnZLSaDsQYX+lSKFCgPpNnJMiVDJxw+tHT0RwptIU/Q/7mYYIjwJERlUJUfgNbfAsUGJw9bUg8CGOU0j2Bs/hoRt81xyDxV+7pkjMn1owK1ygXeZAGf97nzcf9xo/6AVt+IFQQHA2oh+4v6Dz7tsuXbktO0Lik6RgRUQHLfe0ThW6ganch7lFOtQBGclNTmPc9A3/nTKGYdXQzLYFnP6sMFnh80RU5M= ceph-27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:40:57.322 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-16T08:40:57.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1106768145' entity='client.admin' 2024-09-16T08:40:57.572 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:40:58.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.271+0000 7f01bbc14640 1 -- 172.21.15.160:0/3017656615 >> v1:172.21.15.160:6789/0 conn(0x7f01b4102340 legacy=0x7f01b4102740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:58.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.271+0000 7f01bbc14640 1 -- 172.21.15.160:0/3017656615 shutdown_connections 2024-09-16T08:40:58.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.271+0000 7f01bbc14640 1 -- 172.21.15.160:0/3017656615 >> 172.21.15.160:0/3017656615 conn(0x7f01b40fdc10 msgr2=0x7f01b4100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:58.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.271+0000 7f01bbc14640 1 -- 172.21.15.160:0/3017656615 shutdown_connections 2024-09-16T08:40:58.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.271+0000 7f01bbc14640 1 -- 172.21.15.160:0/3017656615 wait complete. 2024-09-16T08:40:58.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.271+0000 7f01bbc14640 1 Processor -- start 2024-09-16T08:40:58.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.271+0000 7f01bbc14640 1 -- start start 2024-09-16T08:40:58.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.271+0000 7f01bbc14640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f01b4077350 con 0x7f01b4102340 2024-09-16T08:40:58.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.272+0000 7f01b99b0640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f01b4102340 0x7f01b4076bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:42194/0 (socket says 172.21.15.160:42194) 2024-09-16T08:40:58.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.272+0000 7f01b99b0640 1 -- 172.21.15.160:0/3305814419 learned_addr learned my addr 172.21.15.160:0/3305814419 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:40:58.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.272+0000 7f01aaffd640 1 -- 172.21.15.160:0/3305814419 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 456151521 0 0) 0x7f01b4077350 con 0x7f01b4102340 2024-09-16T08:40:58.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.272+0000 7f01aaffd640 1 -- 172.21.15.160:0/3305814419 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01a00054b0 con 0x7f01b4102340 2024-09-16T08:40:58.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.272+0000 7f01aaffd640 1 -- 172.21.15.160:0/3305814419 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2721776723 0 0) 0x7f01a00054b0 con 0x7f01b4102340 2024-09-16T08:40:58.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.272+0000 7f01aaffd640 1 -- 172.21.15.160:0/3305814419 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f01b4077350 con 0x7f01b4102340 2024-09-16T08:40:58.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.272+0000 7f01aaffd640 1 -- 172.21.15.160:0/3305814419 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f01b00043e0 con 0x7f01b4102340 2024-09-16T08:40:58.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.272+0000 7f01aaffd640 1 -- 172.21.15.160:0/3305814419 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 974335138 0 0) 0x7f01b4077350 con 0x7f01b4102340 2024-09-16T08:40:58.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.272+0000 7f01aaffd640 1 -- 172.21.15.160:0/3305814419 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b419de40 con 0x7f01b4102340 2024-09-16T08:40:58.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.272+0000 7f01bbc14640 1 -- 172.21.15.160:0/3305814419 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f01b419ce90 con 0x7f01b4102340 2024-09-16T08:40:58.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.272+0000 7f01bbc14640 1 -- 172.21.15.160:0/3305814419 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f01b419d3d0 con 0x7f01b4102340 2024-09-16T08:40:58.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.272+0000 7f01aaffd640 1 -- 172.21.15.160:0/3305814419 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f01b0001ad0 con 0x7f01b4102340 2024-09-16T08:40:58.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.272+0000 7f01aaffd640 1 -- 172.21.15.160:0/3305814419 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f01b0004e50 con 0x7f01b4102340 2024-09-16T08:40:58.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.273+0000 7f01bbc14640 1 -- 172.21.15.160:0/3305814419 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01980050b0 con 0x7f01b4102340 2024-09-16T08:40:58.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.273+0000 7f01aaffd640 1 -- 172.21.15.160:0/3305814419 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 13) ==== 51787+0+0 (unknown 1985690445 0 0) 0x7f01b0005090 con 0x7f01b4102340 2024-09-16T08:40:58.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.274+0000 7f01aaffd640 1 -- 172.21.15.160:0/3305814419 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 2761826324 0 0) 0x7f01b004fa40 con 0x7f01b4102340 2024-09-16T08:40:58.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.278+0000 7f01aaffd640 1 -- 172.21.15.160:0/3305814419 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f01b0018ef0 con 0x7f01b4102340 2024-09-16T08:40:58.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.460+0000 7f01bbc14640 1 -- 172.21.15.160:0/3305814419 --> v1:172.21.15.160:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f01980053a0 con 0x7f01b4102340 2024-09-16T08:40:58.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.462+0000 7f01aaffd640 1 -- 172.21.15.160:0/3305814419 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (unknown 3028693289 0 0) 0x7f01b0003ce0 con 0x7f01b4102340 2024-09-16T08:40:58.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.464+0000 7f01bbc14640 1 -- 172.21.15.160:0/3305814419 >> v1:172.21.15.160:6800/1286008384 conn(0x7f01a00407d0 legacy=0x7f01a0042c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:58.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.464+0000 7f01bbc14640 1 -- 172.21.15.160:0/3305814419 >> v1:172.21.15.160:6789/0 conn(0x7f01b4102340 legacy=0x7f01b4076bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:40:58.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.465+0000 7f01bbc14640 1 -- 172.21.15.160:0/3305814419 shutdown_connections 2024-09-16T08:40:58.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.465+0000 7f01bbc14640 1 -- 172.21.15.160:0/3305814419 >> 172.21.15.160:0/3305814419 conn(0x7f01b40fdc10 msgr2=0x7f01b4100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:40:58.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.465+0000 7f01bbc14640 1 -- 172.21.15.160:0/3305814419 shutdown_connections 2024-09-16T08:40:58.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:40:58.465+0000 7f01bbc14640 1 -- 172.21.15.160:0/3305814419 wait complete. 2024-09-16T08:40:59.358 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-16T08:40:59.358 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-16T08:40:59.461 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:59 smithi160 ceph-mon[30137]: mgrmap e13: y(active, since 12s) 2024-09-16T08:40:59.461 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3305814419' entity='client.admin' 2024-09-16T08:40:59.461 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:59 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:40:59.461 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:59 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:40:59.461 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:59 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi160", "name": "osd_memory_target"} : dispatch 2024-09-16T08:40:59.461 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:59 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:40:59.461 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:59 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:40:59.461 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:40:59 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:40:59.855 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:41:00.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.406+0000 7f0f80ddf640 1 -- 172.21.15.160:0/4085396207 >> v1:172.21.15.160:6789/0 conn(0x7f0f7c102300 legacy=0x7f0f7c102700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:00.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.406+0000 7f0f80ddf640 1 -- 172.21.15.160:0/4085396207 shutdown_connections 2024-09-16T08:41:00.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.406+0000 7f0f80ddf640 1 -- 172.21.15.160:0/4085396207 >> 172.21.15.160:0/4085396207 conn(0x7f0f7c0fdbd0 msgr2=0x7f0f7c100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:00.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.406+0000 7f0f80ddf640 1 -- 172.21.15.160:0/4085396207 shutdown_connections 2024-09-16T08:41:00.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.407+0000 7f0f80ddf640 1 -- 172.21.15.160:0/4085396207 wait complete. 2024-09-16T08:41:00.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.408+0000 7f0f80ddf640 1 Processor -- start 2024-09-16T08:41:00.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.408+0000 7f0f80ddf640 1 -- start start 2024-09-16T08:41:00.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.408+0000 7f0f80ddf640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f0f7c1975e0 con 0x7f0f7c102300 2024-09-16T08:41:00.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.408+0000 7f0f7a59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f0f7c102300 0x7f0f7c196e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:42204/0 (socket says 172.21.15.160:42204) 2024-09-16T08:41:00.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.408+0000 7f0f7a59c640 1 -- 172.21.15.160:0/447050197 learned_addr learned my addr 172.21.15.160:0/447050197 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:41:00.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.408+0000 7f0f737fe640 1 -- 172.21.15.160:0/447050197 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1337678966 0 0) 0x7f0f7c1975e0 con 0x7f0f7c102300 2024-09-16T08:41:00.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.409+0000 7f0f737fe640 1 -- 172.21.15.160:0/447050197 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f680054b0 con 0x7f0f7c102300 2024-09-16T08:41:00.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.409+0000 7f0f737fe640 1 -- 172.21.15.160:0/447050197 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 479098781 0 0) 0x7f0f680054b0 con 0x7f0f7c102300 2024-09-16T08:41:00.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.409+0000 7f0f737fe640 1 -- 172.21.15.160:0/447050197 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0f7c1975e0 con 0x7f0f7c102300 2024-09-16T08:41:00.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.409+0000 7f0f737fe640 1 -- 172.21.15.160:0/447050197 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f0f6c0043e0 con 0x7f0f7c102300 2024-09-16T08:41:00.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.409+0000 7f0f737fe640 1 -- 172.21.15.160:0/447050197 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4061691192 0 0) 0x7f0f7c1975e0 con 0x7f0f7c102300 2024-09-16T08:41:00.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.409+0000 7f0f737fe640 1 -- 172.21.15.160:0/447050197 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7c1977d0 con 0x7f0f7c102300 2024-09-16T08:41:00.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.409+0000 7f0f80ddf640 1 -- 172.21.15.160:0/447050197 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7c19b0b0 con 0x7f0f7c102300 2024-09-16T08:41:00.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.409+0000 7f0f80ddf640 1 -- 172.21.15.160:0/447050197 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0f7c19b4d0 con 0x7f0f7c102300 2024-09-16T08:41:00.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.409+0000 7f0f737fe640 1 -- 172.21.15.160:0/447050197 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f0f6c001ad0 con 0x7f0f7c102300 2024-09-16T08:41:00.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.409+0000 7f0f737fe640 1 -- 172.21.15.160:0/447050197 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f0f6c004de0 con 0x7f0f7c102300 2024-09-16T08:41:00.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.410+0000 7f0f737fe640 1 -- 172.21.15.160:0/447050197 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 13) ==== 51787+0+0 (unknown 1985690445 0 0) 0x7f0f6c005020 con 0x7f0f7c102300 2024-09-16T08:41:00.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.410+0000 7f0f80ddf640 1 -- 172.21.15.160:0/447050197 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f5c0050b0 con 0x7f0f7c102300 2024-09-16T08:41:00.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.410+0000 7f0f737fe640 1 -- 172.21.15.160:0/447050197 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 2761826324 0 0) 0x7f0f6c04f820 con 0x7f0f7c102300 2024-09-16T08:41:00.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.415+0000 7f0f737fe640 1 -- 172.21.15.160:0/447050197 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f0f6c018e80 con 0x7f0f7c102300 2024-09-16T08:41:00.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.603+0000 7f0f80ddf640 1 -- 172.21.15.160:0/447050197 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f0f5c000bc0 con 0x7f0f680407b0 2024-09-16T08:41:00.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.607+0000 7f0f737fe640 1 -- 172.21.15.160:0/447050197 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (unknown 0 0 0) 0x7f0f5c000bc0 con 0x7f0f680407b0 2024-09-16T08:41:00.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.610+0000 7f0f80ddf640 1 -- 172.21.15.160:0/447050197 >> v1:172.21.15.160:6800/1286008384 conn(0x7f0f680407b0 legacy=0x7f0f68042c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:00.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.610+0000 7f0f80ddf640 1 -- 172.21.15.160:0/447050197 >> v1:172.21.15.160:6789/0 conn(0x7f0f7c102300 legacy=0x7f0f7c196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:00.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.610+0000 7f0f80ddf640 1 -- 172.21.15.160:0/447050197 shutdown_connections 2024-09-16T08:41:00.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.610+0000 7f0f80ddf640 1 -- 172.21.15.160:0/447050197 >> 172.21.15.160:0/447050197 conn(0x7f0f7c0fdbd0 msgr2=0x7f0f7c100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:00.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.611+0000 7f0f80ddf640 1 -- 172.21.15.160:0/447050197 shutdown_connections 2024-09-16T08:41:00.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:00.611+0000 7f0f80ddf640 1 -- 172.21.15.160:0/447050197 wait complete. 2024-09-16T08:41:01.720 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:01 smithi160 ceph-mon[30137]: from='client.14176 v1:172.21.15.160:0/447050197' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:41:01.720 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:01 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:01.720 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:01 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:01.720 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:01 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:01.720 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:01 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:41:01.720 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:01 smithi160 ceph-mon[30137]: Updating smithi160:/etc/ceph/ceph.conf 2024-09-16T08:41:01.720 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:01 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:01.720 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:01 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:01.720 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:01 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:01.722 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi169 2024-09-16T08:41:01.722 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T08:41:01.722 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.conf 2024-09-16T08:41:01.741 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T08:41:01.742 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:41:01.800 INFO:tasks.cephadm:Adding host smithi169 to orchestrator... 2024-09-16T08:41:01.800 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch host add smithi169 2024-09-16T08:41:02.044 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:41:02.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.519+0000 7f3afa42a640 1 -- 172.21.15.160:0/29629574 >> v1:172.21.15.160:6789/0 conn(0x7f3af41015c0 legacy=0x7f3af41019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:02.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.519+0000 7f3afa42a640 1 -- 172.21.15.160:0/29629574 shutdown_connections 2024-09-16T08:41:02.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.519+0000 7f3afa42a640 1 -- 172.21.15.160:0/29629574 >> 172.21.15.160:0/29629574 conn(0x7f3af40fd010 msgr2=0x7f3af40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:02.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.519+0000 7f3afa42a640 1 -- 172.21.15.160:0/29629574 shutdown_connections 2024-09-16T08:41:02.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.519+0000 7f3afa42a640 1 -- 172.21.15.160:0/29629574 wait complete. 2024-09-16T08:41:02.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.520+0000 7f3afa42a640 1 Processor -- start 2024-09-16T08:41:02.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.520+0000 7f3afa42a640 1 -- start start 2024-09-16T08:41:02.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.520+0000 7f3afa42a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f3af40ff2b0 con 0x7f3af41015c0 2024-09-16T08:41:02.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.520+0000 7f3af3fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f3af41015c0 0x7f3af40fd510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:42224/0 (socket says 172.21.15.160:42224) 2024-09-16T08:41:02.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.520+0000 7f3af3fff640 1 -- 172.21.15.160:0/2117209265 learned_addr learned my addr 172.21.15.160:0/2117209265 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:41:02.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.520+0000 7f3af17fa640 1 -- 172.21.15.160:0/2117209265 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3235561292 0 0) 0x7f3af40ff2b0 con 0x7f3af41015c0 2024-09-16T08:41:02.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.520+0000 7f3af17fa640 1 -- 172.21.15.160:0/2117209265 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3adc003520 con 0x7f3af41015c0 2024-09-16T08:41:02.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.521+0000 7f3af17fa640 1 -- 172.21.15.160:0/2117209265 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2340953579 0 0) 0x7f3adc003520 con 0x7f3af41015c0 2024-09-16T08:41:02.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.521+0000 7f3af17fa640 1 -- 172.21.15.160:0/2117209265 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3af40ff2b0 con 0x7f3af41015c0 2024-09-16T08:41:02.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.521+0000 7f3af17fa640 1 -- 172.21.15.160:0/2117209265 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f3ae40013b0 con 0x7f3af41015c0 2024-09-16T08:41:02.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.521+0000 7f3af17fa640 1 -- 172.21.15.160:0/2117209265 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1685480904 0 0) 0x7f3af40ff2b0 con 0x7f3af41015c0 2024-09-16T08:41:02.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.521+0000 7f3af17fa640 1 -- 172.21.15.160:0/2117209265 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af40fdc20 con 0x7f3af41015c0 2024-09-16T08:41:02.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.521+0000 7f3afa42a640 1 -- 172.21.15.160:0/2117209265 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3af40fdf10 con 0x7f3af41015c0 2024-09-16T08:41:02.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.521+0000 7f3afa42a640 1 -- 172.21.15.160:0/2117209265 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f3af41a2340 con 0x7f3af41015c0 2024-09-16T08:41:02.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.521+0000 7f3af17fa640 1 -- 172.21.15.160:0/2117209265 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f3ae40011d0 con 0x7f3af41015c0 2024-09-16T08:41:02.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.521+0000 7f3af17fa640 1 -- 172.21.15.160:0/2117209265 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f3ae4004ac0 con 0x7f3af41015c0 2024-09-16T08:41:02.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.522+0000 7f3af17fa640 1 -- 172.21.15.160:0/2117209265 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 13) ==== 51787+0+0 (unknown 1985690445 0 0) 0x7f3ae4004c80 con 0x7f3af41015c0 2024-09-16T08:41:02.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.522+0000 7f3afa42a640 1 -- 172.21.15.160:0/2117209265 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae00050b0 con 0x7f3af41015c0 2024-09-16T08:41:02.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.522+0000 7f3af17fa640 1 -- 172.21.15.160:0/2117209265 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 2761826324 0 0) 0x7f3ae404f550 con 0x7f3af41015c0 2024-09-16T08:41:02.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.527+0000 7f3af17fa640 1 -- 172.21.15.160:0/2117209265 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f3ae4005530 con 0x7f3af41015c0 2024-09-16T08:41:02.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:02 smithi160 ceph-mon[30137]: Updating smithi160:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:41:02.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:02 smithi160 ceph-mon[30137]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:41:02.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:02 smithi160 ceph-mon[30137]: Updating smithi160:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:41:02.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:02.709+0000 7f3afa42a640 1 -- 172.21.15.160:0/2117209265 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}) -- 0x7f3ae0000bc0 con 0x7f3adc0408e0 2024-09-16T08:41:03.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:03 smithi160 ceph-mon[30137]: from='client.14178 v1:172.21.15.160:0/2117209265' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:41:04.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:04 smithi160 ceph-mon[30137]: Deploying cephadm binary to smithi169 2024-09-16T08:41:05.762 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:05.761+0000 7f3af17fa640 1 -- 172.21.15.160:0/2117209265 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (unknown 0 0 4043436670) 0x7f3ae0000bc0 con 0x7f3adc0408e0 2024-09-16T08:41:05.762 INFO:teuthology.orchestra.run.smithi160.stdout:Added host 'smithi169' with addr '172.21.15.169' 2024-09-16T08:41:05.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:05.763+0000 7f3afa42a640 1 -- 172.21.15.160:0/2117209265 >> v1:172.21.15.160:6800/1286008384 conn(0x7f3adc0408e0 legacy=0x7f3adc042da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:05.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:05.763+0000 7f3afa42a640 1 -- 172.21.15.160:0/2117209265 >> v1:172.21.15.160:6789/0 conn(0x7f3af41015c0 legacy=0x7f3af40fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:05.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:05.763+0000 7f3afa42a640 1 -- 172.21.15.160:0/2117209265 shutdown_connections 2024-09-16T08:41:05.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:05.763+0000 7f3afa42a640 1 -- 172.21.15.160:0/2117209265 >> 172.21.15.160:0/2117209265 conn(0x7f3af40fd010 msgr2=0x7f3af406ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:05.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:05.764+0000 7f3afa42a640 1 -- 172.21.15.160:0/2117209265 shutdown_connections 2024-09-16T08:41:05.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:05.764+0000 7f3afa42a640 1 -- 172.21.15.160:0/2117209265 wait complete. 2024-09-16T08:41:06.394 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T08:41:06.635 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:41:07.062 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:06 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:07.062 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:06 smithi160 ceph-mon[30137]: Added host smithi169 2024-09-16T08:41:07.062 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:06 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:07.062 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:06 smithi160 ceph-mon[30137]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:07.062 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:06 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:07.062 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:06 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:07.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.144+0000 7feda7725640 1 -- 172.21.15.160:0/2358973799 >> v1:172.21.15.160:6789/0 conn(0x7feda0103470 legacy=0x7feda0103850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:07.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.144+0000 7feda7725640 1 -- 172.21.15.160:0/2358973799 shutdown_connections 2024-09-16T08:41:07.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.144+0000 7feda7725640 1 -- 172.21.15.160:0/2358973799 >> 172.21.15.160:0/2358973799 conn(0x7feda0078170 msgr2=0x7feda00ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:07.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.144+0000 7feda7725640 1 -- 172.21.15.160:0/2358973799 shutdown_connections 2024-09-16T08:41:07.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.144+0000 7feda7725640 1 -- 172.21.15.160:0/2358973799 wait complete. 2024-09-16T08:41:07.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.145+0000 7feda7725640 1 Processor -- start 2024-09-16T08:41:07.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.145+0000 7feda7725640 1 -- start start 2024-09-16T08:41:07.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.145+0000 7feda7725640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7feda01978b0 con 0x7feda0103470 2024-09-16T08:41:07.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.145+0000 7feda54c1640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7feda0103470 0x7feda01971a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:56568/0 (socket says 172.21.15.160:56568) 2024-09-16T08:41:07.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.145+0000 7feda54c1640 1 -- 172.21.15.160:0/2525914473 learned_addr learned my addr 172.21.15.160:0/2525914473 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:41:07.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.145+0000 7fed967fc640 1 -- 172.21.15.160:0/2525914473 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1112562742 0 0) 0x7feda01978b0 con 0x7feda0103470 2024-09-16T08:41:07.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.145+0000 7fed967fc640 1 -- 172.21.15.160:0/2525914473 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed8c003520 con 0x7feda0103470 2024-09-16T08:41:07.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.145+0000 7fed967fc640 1 -- 172.21.15.160:0/2525914473 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2878210931 0 0) 0x7fed8c003520 con 0x7feda0103470 2024-09-16T08:41:07.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.145+0000 7fed967fc640 1 -- 172.21.15.160:0/2525914473 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7feda01978b0 con 0x7feda0103470 2024-09-16T08:41:07.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.145+0000 7fed967fc640 1 -- 172.21.15.160:0/2525914473 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fed900013b0 con 0x7feda0103470 2024-09-16T08:41:07.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.145+0000 7fed967fc640 1 -- 172.21.15.160:0/2525914473 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2465270437 0 0) 0x7feda01978b0 con 0x7feda0103470 2024-09-16T08:41:07.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.145+0000 7fed967fc640 1 -- 172.21.15.160:0/2525914473 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda0197a80 con 0x7feda0103470 2024-09-16T08:41:07.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.145+0000 7feda7725640 1 -- 172.21.15.160:0/2525914473 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7feda0197d70 con 0x7feda0103470 2024-09-16T08:41:07.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.146+0000 7feda7725640 1 -- 172.21.15.160:0/2525914473 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7feda019fee0 con 0x7feda0103470 2024-09-16T08:41:07.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.146+0000 7fed967fc640 1 -- 172.21.15.160:0/2525914473 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fed900011d0 con 0x7feda0103470 2024-09-16T08:41:07.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.146+0000 7fed967fc640 1 -- 172.21.15.160:0/2525914473 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fed90004b90 con 0x7feda0103470 2024-09-16T08:41:07.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.146+0000 7fed967fc640 1 -- 172.21.15.160:0/2525914473 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 13) ==== 51787+0+0 (unknown 1985690445 0 0) 0x7fed90004d50 con 0x7feda0103470 2024-09-16T08:41:07.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.147+0000 7feda7725640 1 -- 172.21.15.160:0/2525914473 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed840050b0 con 0x7feda0103470 2024-09-16T08:41:07.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.147+0000 7fed967fc640 1 -- 172.21.15.160:0/2525914473 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 2761826324 0 0) 0x7fed9004f610 con 0x7feda0103470 2024-09-16T08:41:07.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.151+0000 7fed967fc640 1 -- 172.21.15.160:0/2525914473 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fed90018c60 con 0x7feda0103470 2024-09-16T08:41:07.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.332+0000 7feda7725640 1 -- 172.21.15.160:0/2525914473 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed84000bc0 con 0x7fed8c040890 2024-09-16T08:41:07.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.333+0000 7fed967fc640 1 -- 172.21.15.160:0/2525914473 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (unknown 0 0 1163582761) 0x7fed84000bc0 con 0x7fed8c040890 2024-09-16T08:41:07.335 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:41:07.335 INFO:teuthology.orchestra.run.smithi160.stdout:[{"addr": "172.21.15.160", "hostname": "smithi160", "labels": [], "status": ""}, {"addr": "172.21.15.169", "hostname": "smithi169", "labels": [], "status": ""}] 2024-09-16T08:41:07.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.336+0000 7feda7725640 1 -- 172.21.15.160:0/2525914473 >> v1:172.21.15.160:6800/1286008384 conn(0x7fed8c040890 legacy=0x7fed8c042d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:07.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.336+0000 7feda7725640 1 -- 172.21.15.160:0/2525914473 >> v1:172.21.15.160:6789/0 conn(0x7feda0103470 legacy=0x7feda01971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:07.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.336+0000 7feda7725640 1 -- 172.21.15.160:0/2525914473 shutdown_connections 2024-09-16T08:41:07.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.336+0000 7feda7725640 1 -- 172.21.15.160:0/2525914473 >> 172.21.15.160:0/2525914473 conn(0x7feda0078170 msgr2=0x7feda010a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:07.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.336+0000 7feda7725640 1 -- 172.21.15.160:0/2525914473 shutdown_connections 2024-09-16T08:41:07.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:07.336+0000 7feda7725640 1 -- 172.21.15.160:0/2525914473 wait complete. 2024-09-16T08:41:07.968 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-16T08:41:07.969 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-16T08:41:08.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:08 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:08.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:08 smithi160 ceph-mon[30137]: from='client.14180 v1:172.21.15.160:0/2525914473' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:41:08.212 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:41:08.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.614+0000 7fe303994640 1 -- 172.21.15.160:0/1131196332 >> v1:172.21.15.160:6789/0 conn(0x7fe2fc1015c0 legacy=0x7fe2fc1019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:08.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.614+0000 7fe303994640 1 -- 172.21.15.160:0/1131196332 shutdown_connections 2024-09-16T08:41:08.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.614+0000 7fe303994640 1 -- 172.21.15.160:0/1131196332 >> 172.21.15.160:0/1131196332 conn(0x7fe2fc0fd010 msgr2=0x7fe2fc0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:08.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.614+0000 7fe303994640 1 -- 172.21.15.160:0/1131196332 shutdown_connections 2024-09-16T08:41:08.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.614+0000 7fe303994640 1 -- 172.21.15.160:0/1131196332 wait complete. 2024-09-16T08:41:08.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.615+0000 7fe303994640 1 Processor -- start 2024-09-16T08:41:08.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.615+0000 7fe303994640 1 -- start start 2024-09-16T08:41:08.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.615+0000 7fe303994640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe2fc1979c0 con 0x7fe2fc1015c0 2024-09-16T08:41:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.615+0000 7fe301730640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fe2fc1015c0 0x7fe2fc1972b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:56584/0 (socket says 172.21.15.160:56584) 2024-09-16T08:41:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.615+0000 7fe301730640 1 -- 172.21.15.160:0/2948640741 learned_addr learned my addr 172.21.15.160:0/2948640741 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:41:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.616+0000 7fe2ee7fc640 1 -- 172.21.15.160:0/2948640741 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2279201724 0 0) 0x7fe2fc1979c0 con 0x7fe2fc1015c0 2024-09-16T08:41:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.616+0000 7fe2ee7fc640 1 -- 172.21.15.160:0/2948640741 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2e80054b0 con 0x7fe2fc1015c0 2024-09-16T08:41:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.616+0000 7fe2ee7fc640 1 -- 172.21.15.160:0/2948640741 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3223430708 0 0) 0x7fe2e80054b0 con 0x7fe2fc1015c0 2024-09-16T08:41:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.616+0000 7fe2ee7fc640 1 -- 172.21.15.160:0/2948640741 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe2fc1979c0 con 0x7fe2fc1015c0 2024-09-16T08:41:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.616+0000 7fe2ee7fc640 1 -- 172.21.15.160:0/2948640741 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fe2f8001960 con 0x7fe2fc1015c0 2024-09-16T08:41:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.616+0000 7fe2ee7fc640 1 -- 172.21.15.160:0/2948640741 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1102981692 0 0) 0x7fe2fc1979c0 con 0x7fe2fc1015c0 2024-09-16T08:41:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.616+0000 7fe2ee7fc640 1 -- 172.21.15.160:0/2948640741 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2fc197b90 con 0x7fe2fc1015c0 2024-09-16T08:41:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.616+0000 7fe303994640 1 -- 172.21.15.160:0/2948640741 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe2fc197e80 con 0x7fe2fc1015c0 2024-09-16T08:41:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.616+0000 7fe303994640 1 -- 172.21.15.160:0/2948640741 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe2fc19ff70 con 0x7fe2fc1015c0 2024-09-16T08:41:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.616+0000 7fe2ee7fc640 1 -- 172.21.15.160:0/2948640741 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fe2f8001d10 con 0x7fe2fc1015c0 2024-09-16T08:41:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.616+0000 7fe2ee7fc640 1 -- 172.21.15.160:0/2948640741 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fe2f8004c10 con 0x7fe2fc1015c0 2024-09-16T08:41:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.617+0000 7fe303994640 1 -- 172.21.15.160:0/2948640741 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2e00050b0 con 0x7fe2fc1015c0 2024-09-16T08:41:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.617+0000 7fe2ee7fc640 1 -- 172.21.15.160:0/2948640741 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 13) ==== 51787+0+0 (unknown 1985690445 0 0) 0x7fe2f8004dd0 con 0x7fe2fc1015c0 2024-09-16T08:41:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.617+0000 7fe2ee7fc640 1 -- 172.21.15.160:0/2948640741 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 2761826324 0 0) 0x7fe2f804f4f0 con 0x7fe2fc1015c0 2024-09-16T08:41:08.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.622+0000 7fe2ee7fc640 1 -- 172.21.15.160:0/2948640741 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fe2f8018c90 con 0x7fe2fc1015c0 2024-09-16T08:41:08.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:08.801+0000 7fe303994640 1 -- 172.21.15.160:0/2948640741 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fe2e00053a0 con 0x7fe2fc1015c0 2024-09-16T08:41:09.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:09.088+0000 7fe2ee7fc640 1 -- 172.21.15.160:0/2948640741 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (unknown 3126668360 0 0) 0x7fe2f8021e90 con 0x7fe2fc1015c0 2024-09-16T08:41:09.090 INFO:teuthology.orchestra.run.smithi160.stderr:adjusted tunables profile to default 2024-09-16T08:41:09.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:09.090+0000 7fe303994640 1 -- 172.21.15.160:0/2948640741 >> v1:172.21.15.160:6800/1286008384 conn(0x7fe2e80407d0 legacy=0x7fe2e8042c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:09.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:09.090+0000 7fe303994640 1 -- 172.21.15.160:0/2948640741 >> v1:172.21.15.160:6789/0 conn(0x7fe2fc1015c0 legacy=0x7fe2fc1972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:09.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:09.091+0000 7fe303994640 1 -- 172.21.15.160:0/2948640741 shutdown_connections 2024-09-16T08:41:09.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:09.091+0000 7fe303994640 1 -- 172.21.15.160:0/2948640741 >> 172.21.15.160:0/2948640741 conn(0x7fe2fc0fd010 msgr2=0x7fe2fc06ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:09.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:09.091+0000 7fe303994640 1 -- 172.21.15.160:0/2948640741 shutdown_connections 2024-09-16T08:41:09.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:09.091+0000 7fe303994640 1 -- 172.21.15.160:0/2948640741 wait complete. 2024-09-16T08:41:09.403 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:09 smithi160 ceph-mon[30137]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:09.404 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2948640741' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-16T08:41:09.713 INFO:tasks.cephadm:Adding mon.a on smithi160 2024-09-16T08:41:09.713 INFO:tasks.cephadm:Adding mon.c on smithi160 2024-09-16T08:41:09.713 INFO:tasks.cephadm:Adding mon.b on smithi169 2024-09-16T08:41:09.713 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi160:[v1:172.21.15.160:6789]=a;smithi160:[v1:172.21.15.160:6790]=c;smithi169:[v1:172.21.15.169:6789]=b' 2024-09-16T08:41:09.958 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:41:10.003 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:41:10.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2948640741' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T08:41:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:10 smithi160 ceph-mon[30137]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T08:41:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:10 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:10 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:10 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:10 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:10 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"} : dispatch 2024-09-16T08:41:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:10 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:10 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:41:10.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.510+0000 7fa5d0b2f640 1 -- 172.21.15.169:0/2125302880 >> v1:172.21.15.160:6789/0 conn(0x7fa5cc1011f0 legacy=0x7fa5cc1015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:10.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.510+0000 7fa5d0b2f640 1 -- 172.21.15.169:0/2125302880 shutdown_connections 2024-09-16T08:41:10.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.510+0000 7fa5d0b2f640 1 -- 172.21.15.169:0/2125302880 >> 172.21.15.169:0/2125302880 conn(0x7fa5cc0fcae0 msgr2=0x7fa5cc0fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:10.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.511+0000 7fa5d0b2f640 1 -- 172.21.15.169:0/2125302880 shutdown_connections 2024-09-16T08:41:10.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.511+0000 7fa5d0b2f640 1 -- 172.21.15.169:0/2125302880 wait complete. 2024-09-16T08:41:10.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.511+0000 7fa5d0b2f640 1 Processor -- start 2024-09-16T08:41:10.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.511+0000 7fa5d0b2f640 1 -- start start 2024-09-16T08:41:10.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.511+0000 7fa5d0b2f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa5cc19ab50 con 0x7fa5cc1011f0 2024-09-16T08:41:10.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.512+0000 7fa5ca59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fa5cc1011f0 0x7fa5cc06ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.169:57830/0 (socket says 172.21.15.169:57830) 2024-09-16T08:41:10.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.512+0000 7fa5ca59c640 1 -- 172.21.15.169:0/403836414 learned_addr learned my addr 172.21.15.169:0/403836414 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:41:10.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.512+0000 7fa5c37fe640 1 -- 172.21.15.169:0/403836414 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 445894484 0 0) 0x7fa5cc19ab50 con 0x7fa5cc1011f0 2024-09-16T08:41:10.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.512+0000 7fa5c37fe640 1 -- 172.21.15.169:0/403836414 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5b80054b0 con 0x7fa5cc1011f0 2024-09-16T08:41:10.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.512+0000 7fa5c37fe640 1 -- 172.21.15.169:0/403836414 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1474962749 0 0) 0x7fa5b80054b0 con 0x7fa5cc1011f0 2024-09-16T08:41:10.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.512+0000 7fa5c37fe640 1 -- 172.21.15.169:0/403836414 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa5cc19ab50 con 0x7fa5cc1011f0 2024-09-16T08:41:10.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.512+0000 7fa5c37fe640 1 -- 172.21.15.169:0/403836414 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fa5bc0043e0 con 0x7fa5cc1011f0 2024-09-16T08:41:10.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.512+0000 7fa5c37fe640 1 -- 172.21.15.169:0/403836414 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1500371286 0 0) 0x7fa5cc19ab50 con 0x7fa5cc1011f0 2024-09-16T08:41:10.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.512+0000 7fa5c37fe640 1 -- 172.21.15.169:0/403836414 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc19bd30 con 0x7fa5cc1011f0 2024-09-16T08:41:10.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.512+0000 7fa5d0b2f640 1 -- 172.21.15.169:0/403836414 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa5cc19ad20 con 0x7fa5cc1011f0 2024-09-16T08:41:10.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.512+0000 7fa5d0b2f640 1 -- 172.21.15.169:0/403836414 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa5cc19b2b0 con 0x7fa5cc1011f0 2024-09-16T08:41:10.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.512+0000 7fa5c37fe640 1 -- 172.21.15.169:0/403836414 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fa5bc001ad0 con 0x7fa5cc1011f0 2024-09-16T08:41:10.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.512+0000 7fa5c37fe640 1 -- 172.21.15.169:0/403836414 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7fa5bc004de0 con 0x7fa5cc1011f0 2024-09-16T08:41:10.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.513+0000 7fa5d0b2f640 1 -- 172.21.15.169:0/403836414 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5ac0050b0 con 0x7fa5cc1011f0 2024-09-16T08:41:10.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.513+0000 7fa5c37fe640 1 -- 172.21.15.169:0/403836414 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 13) ==== 51787+0+0 (unknown 1985690445 0 0) 0x7fa5bc005020 con 0x7fa5cc1011f0 2024-09-16T08:41:10.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.514+0000 7fa5c37fe640 1 -- 172.21.15.169:0/403836414 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 3038969552 0 0) 0x7fa5bc04f950 con 0x7fa5cc1011f0 2024-09-16T08:41:10.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.518+0000 7fa5c37fe640 1 -- 172.21.15.169:0/403836414 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fa5bc0168f0 con 0x7fa5cc1011f0 2024-09-16T08:41:10.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.707+0000 7fa5d0b2f640 1 -- 172.21.15.169:0/403836414 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi160:[v1:172.21.15.160:6789]=a;smithi160:[v1:172.21.15.160:6790]=c;smithi169:[v1:172.21.15.169:6789]=b", "target": ["mon-mgr", ""]}) -- 0x7fa5ac000ca0 con 0x7fa5b8040820 2024-09-16T08:41:10.714 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.714+0000 7fa5c37fe640 1 -- 172.21.15.169:0/403836414 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (unknown 0 0 3265049985) 0x7fa5ac000ca0 con 0x7fa5b8040820 2024-09-16T08:41:10.714 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mon update... 2024-09-16T08:41:10.716 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.716+0000 7fa5d0b2f640 1 -- 172.21.15.169:0/403836414 >> v1:172.21.15.160:6800/1286008384 conn(0x7fa5b8040820 legacy=0x7fa5b8042ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:10.716 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.716+0000 7fa5d0b2f640 1 -- 172.21.15.169:0/403836414 >> v1:172.21.15.160:6789/0 conn(0x7fa5cc1011f0 legacy=0x7fa5cc06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:10.716 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.716+0000 7fa5d0b2f640 1 -- 172.21.15.169:0/403836414 shutdown_connections 2024-09-16T08:41:10.716 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.716+0000 7fa5d0b2f640 1 -- 172.21.15.169:0/403836414 >> 172.21.15.169:0/403836414 conn(0x7fa5cc0fcae0 msgr2=0x7fa5cc0feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:10.717 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.716+0000 7fa5d0b2f640 1 -- 172.21.15.169:0/403836414 shutdown_connections 2024-09-16T08:41:10.717 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:10.716+0000 7fa5d0b2f640 1 -- 172.21.15.169:0/403836414 wait complete. 2024-09-16T08:41:11.362 DEBUG:teuthology.orchestra.run.smithi160:mon.c> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mon.c.service 2024-09-16T08:41:11.365 DEBUG:teuthology.orchestra.run.smithi169:mon.b> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mon.b.service 2024-09-16T08:41:11.368 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T08:41:11.368 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:41:11.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:11.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T08:41:11.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: Updating smithi169:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:41:11.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:41:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: Updating smithi169:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:41:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: from='client.14184 v1:172.21.15.169:0/403836414' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi160:[v1:172.21.15.160:6789]=a;smithi160:[v1:172.21.15.160:6790]=c;smithi169:[v1:172.21.15.169:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:41:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: Saving service mon spec with placement smithi160:[v1:172.21.15.160:6789]=a;smithi160:[v1:172.21.15.160:6790]=c;smithi169:[v1:172.21.15.169:6789]=b;count:3 2024-09-16T08:41:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:41:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:41:11.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:11.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:11 smithi160 ceph-mon[30137]: Deploying daemon mon.b on smithi169 2024-09-16T08:41:12.104 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:41:12.519 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.515+0000 7f9fb5a63640 1 -- 172.21.15.169:0/3087964 >> v1:172.21.15.160:6789/0 conn(0x7f9fb01035a0 legacy=0x7f9fb0103980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:12.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.515+0000 7f9fb5a63640 1 -- 172.21.15.169:0/3087964 shutdown_connections 2024-09-16T08:41:12.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.515+0000 7f9fb5a63640 1 -- 172.21.15.169:0/3087964 >> 172.21.15.169:0/3087964 conn(0x7f9fb0078170 msgr2=0x7f9fb00ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:12.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.516+0000 7f9fb5a63640 1 -- 172.21.15.169:0/3087964 shutdown_connections 2024-09-16T08:41:12.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.516+0000 7f9fb5a63640 1 -- 172.21.15.169:0/3087964 wait complete. 2024-09-16T08:41:12.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.516+0000 7f9fb5a63640 1 Processor -- start 2024-09-16T08:41:12.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.516+0000 7f9fb5a63640 1 -- start start 2024-09-16T08:41:12.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.517+0000 7f9fb5a63640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9fb0197900 con 0x7f9fb01035a0 2024-09-16T08:41:12.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.517+0000 7f9faf7fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f9fb01035a0 0x7f9fb01971f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.169:57846/0 (socket says 172.21.15.169:57846) 2024-09-16T08:41:12.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.517+0000 7f9faf7fe640 1 -- 172.21.15.169:0/2599051096 learned_addr learned my addr 172.21.15.169:0/2599051096 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:41:12.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.517+0000 7f9facff9640 1 -- 172.21.15.169:0/2599051096 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3320980130 0 0) 0x7f9fb0197900 con 0x7f9fb01035a0 2024-09-16T08:41:12.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.517+0000 7f9facff9640 1 -- 172.21.15.169:0/2599051096 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f9c0054b0 con 0x7f9fb01035a0 2024-09-16T08:41:12.521 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.517+0000 7f9facff9640 1 -- 172.21.15.169:0/2599051096 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1549327000 0 0) 0x7f9f9c0054b0 con 0x7f9fb01035a0 2024-09-16T08:41:12.521 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.517+0000 7f9facff9640 1 -- 172.21.15.169:0/2599051096 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9fb0197900 con 0x7f9fb01035a0 2024-09-16T08:41:12.521 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.517+0000 7f9facff9640 1 -- 172.21.15.169:0/2599051096 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f9fa0001ce0 con 0x7f9fb01035a0 2024-09-16T08:41:12.521 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.518+0000 7f9facff9640 1 -- 172.21.15.169:0/2599051096 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2702953197 0 0) 0x7f9fb0197900 con 0x7f9fb01035a0 2024-09-16T08:41:12.521 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.518+0000 7f9facff9640 1 -- 172.21.15.169:0/2599051096 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb0197ad0 con 0x7f9fb01035a0 2024-09-16T08:41:12.521 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.518+0000 7f9fb5a63640 1 -- 172.21.15.169:0/2599051096 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb0197dc0 con 0x7f9fb01035a0 2024-09-16T08:41:12.521 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.518+0000 7f9fb5a63640 1 -- 172.21.15.169:0/2599051096 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9fb019ff30 con 0x7f9fb01035a0 2024-09-16T08:41:12.521 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.518+0000 7f9facff9640 1 -- 172.21.15.169:0/2599051096 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f9fa0002090 con 0x7f9fb01035a0 2024-09-16T08:41:12.521 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.518+0000 7f9facff9640 1 -- 172.21.15.169:0/2599051096 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 2368388174 0 0) 0x7f9fa0004f90 con 0x7f9fb01035a0 2024-09-16T08:41:12.521 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.519+0000 7f9fb5a63640 1 -- 172.21.15.169:0/2599051096 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f900050b0 con 0x7f9fb01035a0 2024-09-16T08:41:12.521 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.519+0000 7f9facff9640 1 -- 172.21.15.169:0/2599051096 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 13) ==== 51787+0+0 (unknown 1985690445 0 0) 0x7f9fa00051d0 con 0x7f9fb01035a0 2024-09-16T08:41:12.522 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.519+0000 7f9facff9640 1 -- 172.21.15.169:0/2599051096 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 3038969552 0 0) 0x7f9fa004fa40 con 0x7f9fb01035a0 2024-09-16T08:41:12.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.524+0000 7f9facff9640 1 -- 172.21.15.169:0/2599051096 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f9fa004e2e0 con 0x7f9fb01035a0 2024-09-16T08:41:12.774 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.773+0000 7f9fb5a63640 1 -- 172.21.15.169:0/2599051096 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9f900053a0 con 0x7f9fb01035a0 2024-09-16T08:41:12.774 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.774+0000 7f9facff9640 1 -- 172.21.15.169:0/2599051096 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+697 (unknown 2237029548 0 2424261585) 0x7f9fa004e660 con 0x7f9fb01035a0 2024-09-16T08:41:12.775 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:41:12.775 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"27cca2fc-7407-11ef-bceb-c7b262605968","modified":"2024-09-16T08:39:50.898512Z","created":"2024-09-16T08:39:50.898512Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:41:12.775 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-09-16T08:41:12.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.776+0000 7f9fb5a63640 1 -- 172.21.15.169:0/2599051096 >> v1:172.21.15.160:6800/1286008384 conn(0x7f9f9c040780 legacy=0x7f9f9c042c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:12.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.776+0000 7f9fb5a63640 1 -- 172.21.15.169:0/2599051096 >> v1:172.21.15.160:6789/0 conn(0x7f9fb01035a0 legacy=0x7f9fb01971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:12.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.777+0000 7f9fb5a63640 1 -- 172.21.15.169:0/2599051096 shutdown_connections 2024-09-16T08:41:12.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.777+0000 7f9fb5a63640 1 -- 172.21.15.169:0/2599051096 >> 172.21.15.169:0/2599051096 conn(0x7f9fb0078170 msgr2=0x7f9fb0108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:12.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.777+0000 7f9fb5a63640 1 -- 172.21.15.169:0/2599051096 shutdown_connections 2024-09-16T08:41:12.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:12.777+0000 7f9fb5a63640 1 -- 172.21.15.169:0/2599051096 wait complete. 2024-09-16T08:41:13.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:13 smithi160 ceph-mon[30137]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:13.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.169:0/2599051096' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T08:41:15.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:15 smithi160 ceph-mon[30137]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:15.709 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 podman[33427]: 2024-09-16 08:41:15.687263315 +0000 UTC m=+0.337256275 container start 4b357747a229cec57221a812a464350639522901b2c397f7c81db0008a17f211 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T08:41:15.709 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T08:41:15.709 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev), process ceph-mon, pid 7 2024-09-16T08:41:15.710 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: pidfile_write: ignore empty --pid-file 2024-09-16T08:41:15.757 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T08:41:15.757 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:41:16.018 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:41:16.101 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: load: jerasure load: lrc 2024-09-16T08:41:16.101 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: RocksDB version: 7.9.2 2024-09-16T08:41:16.102 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Git sha 0 2024-09-16T08:41:16.102 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Compile date 2024-09-13 14:50:41 2024-09-16T08:41:16.102 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: DB SUMMARY 2024-09-16T08:41:16.102 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: DB Session ID: 2FKTWNAOG9VORB4DA7A5 2024-09-16T08:41:16.102 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: CURRENT file: CURRENT 2024-09-16T08:41:16.102 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T08:41:16.103 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-16T08:41:16.103 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-16T08:41:16.103 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 476 ; 2024-09-16T08:41:16.103 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.error_if_exists: 0 2024-09-16T08:41:16.103 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.create_if_missing: 0 2024-09-16T08:41:16.103 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.paranoid_checks: 1 2024-09-16T08:41:16.104 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T08:41:16.104 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T08:41:16.104 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T08:41:16.104 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.env: 0x55abc6cab120 2024-09-16T08:41:16.104 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T08:41:16.104 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.info_log: 0x55abc8221680 2024-09-16T08:41:16.105 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T08:41:16.105 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.statistics: (nil) 2024-09-16T08:41:16.105 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.use_fsync: 0 2024-09-16T08:41:16.105 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_log_file_size: 0 2024-09-16T08:41:16.105 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T08:41:16.106 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T08:41:16.106 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T08:41:16.106 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T08:41:16.106 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.allow_fallocate: 1 2024-09-16T08:41:16.106 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T08:41:16.106 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T08:41:16.107 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.use_direct_reads: 0 2024-09-16T08:41:16.107 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T08:41:16.107 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T08:41:16.107 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.db_log_dir: 2024-09-16T08:41:16.107 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.wal_dir: 2024-09-16T08:41:16.107 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T08:41:16.108 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T08:41:16.108 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T08:41:16.108 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T08:41:16.108 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T08:41:16.108 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T08:41:16.108 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T08:41:16.109 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T08:41:16.109 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.write_buffer_manager: 0x55abc8225b80 2024-09-16T08:41:16.109 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T08:41:16.109 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T08:41:16.109 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T08:41:16.110 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T08:41:16.110 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T08:41:16.110 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T08:41:16.110 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T08:41:16.110 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T08:41:16.110 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.unordered_write: 0 2024-09-16T08:41:16.111 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T08:41:16.111 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T08:41:16.111 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T08:41:16.111 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T08:41:16.111 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.row_cache: None 2024-09-16T08:41:16.111 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.wal_filter: None 2024-09-16T08:41:16.112 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T08:41:16.112 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T08:41:16.112 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.two_write_queues: 0 2024-09-16T08:41:16.112 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T08:41:16.112 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.wal_compression: 0 2024-09-16T08:41:16.112 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.atomic_flush: 0 2024-09-16T08:41:16.112 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T08:41:16.113 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T08:41:16.113 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T08:41:16.113 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.log_readahead_size: 0 2024-09-16T08:41:16.113 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T08:41:16.113 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T08:41:16.113 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T08:41:16.114 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T08:41:16.114 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T08:41:16.114 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T08:41:16.114 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T08:41:16.114 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_background_jobs: 2 2024-09-16T08:41:16.114 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_background_compactions: -1 2024-09-16T08:41:16.115 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_subcompactions: 1 2024-09-16T08:41:16.115 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T08:41:16.115 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T08:41:16.115 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T08:41:16.115 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T08:41:16.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T08:41:16.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T08:41:16.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T08:41:16.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T08:41:16.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_open_files: -1 2024-09-16T08:41:16.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T08:41:16.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T08:41:16.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T08:41:16.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T08:41:16.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_background_flushes: -1 2024-09-16T08:41:16.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Compression algorithms supported: 2024-09-16T08:41:16.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: kZSTD supported: 0 2024-09-16T08:41:16.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: kXpressCompression supported: 0 2024-09-16T08:41:16.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: kBZip2Compression supported: 0 2024-09-16T08:41:16.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T08:41:16.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: kLZ4Compression supported: 1 2024-09-16T08:41:16.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: kZlibCompression supported: 1 2024-09-16T08:41:16.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T08:41:16.119 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: kSnappyCompression supported: 1 2024-09-16T08:41:16.119 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T08:41:16.119 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T08:41:16.119 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-16T08:41:16.119 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T08:41:16.119 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T08:41:16.120 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.merge_operator: 2024-09-16T08:41:16.120 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compaction_filter: None 2024-09-16T08:41:16.120 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T08:41:16.120 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T08:41:16.120 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T08:41:16.121 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T08:41:16.121 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55abc82212a0) 2024-09-16T08:41:16.121 INFO:journalctl@ceph.mon.b.smithi169.stdout: cache_index_and_filter_blocks: 1 2024-09-16T08:41:16.121 INFO:journalctl@ceph.mon.b.smithi169.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T08:41:16.121 INFO:journalctl@ceph.mon.b.smithi169.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T08:41:16.121 INFO:journalctl@ceph.mon.b.smithi169.stdout: pin_top_level_index_and_filter: 1 2024-09-16T08:41:16.122 INFO:journalctl@ceph.mon.b.smithi169.stdout: index_type: 0 2024-09-16T08:41:16.122 INFO:journalctl@ceph.mon.b.smithi169.stdout: data_block_index_type: 0 2024-09-16T08:41:16.122 INFO:journalctl@ceph.mon.b.smithi169.stdout: index_shortening: 1 2024-09-16T08:41:16.122 INFO:journalctl@ceph.mon.b.smithi169.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T08:41:16.122 INFO:journalctl@ceph.mon.b.smithi169.stdout: checksum: 4 2024-09-16T08:41:16.122 INFO:journalctl@ceph.mon.b.smithi169.stdout: no_block_cache: 0 2024-09-16T08:41:16.122 INFO:journalctl@ceph.mon.b.smithi169.stdout: block_cache: 0x55abc8215610 2024-09-16T08:41:16.122 INFO:journalctl@ceph.mon.b.smithi169.stdout: block_cache_name: BinnedLRUCache 2024-09-16T08:41:16.122 INFO:journalctl@ceph.mon.b.smithi169.stdout: block_cache_options: 2024-09-16T08:41:16.123 INFO:journalctl@ceph.mon.b.smithi169.stdout: capacity : 536870912 2024-09-16T08:41:16.123 INFO:journalctl@ceph.mon.b.smithi169.stdout: num_shard_bits : 4 2024-09-16T08:41:16.123 INFO:journalctl@ceph.mon.b.smithi169.stdout: strict_capacity_limit : 0 2024-09-16T08:41:16.123 INFO:journalctl@ceph.mon.b.smithi169.stdout: high_pri_pool_ratio: 0.000 2024-09-16T08:41:16.123 INFO:journalctl@ceph.mon.b.smithi169.stdout: block_cache_compressed: (nil) 2024-09-16T08:41:16.123 INFO:journalctl@ceph.mon.b.smithi169.stdout: persistent_cache: (nil) 2024-09-16T08:41:16.123 INFO:journalctl@ceph.mon.b.smithi169.stdout: block_size: 4096 2024-09-16T08:41:16.123 INFO:journalctl@ceph.mon.b.smithi169.stdout: block_size_deviation: 10 2024-09-16T08:41:16.124 INFO:journalctl@ceph.mon.b.smithi169.stdout: block_restart_interval: 16 2024-09-16T08:41:16.124 INFO:journalctl@ceph.mon.b.smithi169.stdout: index_block_restart_interval: 1 2024-09-16T08:41:16.124 INFO:journalctl@ceph.mon.b.smithi169.stdout: metadata_block_size: 4096 2024-09-16T08:41:16.125 INFO:journalctl@ceph.mon.b.smithi169.stdout: partition_filters: 0 2024-09-16T08:41:16.125 INFO:journalctl@ceph.mon.b.smithi169.stdout: use_delta_encoding: 1 2024-09-16T08:41:16.125 INFO:journalctl@ceph.mon.b.smithi169.stdout: filter_policy: bloomfilter 2024-09-16T08:41:16.125 INFO:journalctl@ceph.mon.b.smithi169.stdout: whole_key_filtering: 1 2024-09-16T08:41:16.125 INFO:journalctl@ceph.mon.b.smithi169.stdout: verify_compression: 0 2024-09-16T08:41:16.125 INFO:journalctl@ceph.mon.b.smithi169.stdout: read_amp_bytes_per_bit: 0 2024-09-16T08:41:16.125 INFO:journalctl@ceph.mon.b.smithi169.stdout: format_version: 5 2024-09-16T08:41:16.126 INFO:journalctl@ceph.mon.b.smithi169.stdout: enable_index_compression: 1 2024-09-16T08:41:16.126 INFO:journalctl@ceph.mon.b.smithi169.stdout: block_align: 0 2024-09-16T08:41:16.126 INFO:journalctl@ceph.mon.b.smithi169.stdout: max_auto_readahead_size: 262144 2024-09-16T08:41:16.126 INFO:journalctl@ceph.mon.b.smithi169.stdout: prepopulate_block_cache: 0 2024-09-16T08:41:16.126 INFO:journalctl@ceph.mon.b.smithi169.stdout: initial_auto_readahead_size: 8192 2024-09-16T08:41:16.126 INFO:journalctl@ceph.mon.b.smithi169.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T08:41:16.126 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T08:41:16.127 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T08:41:16.127 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compression: NoCompression 2024-09-16T08:41:16.127 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T08:41:16.127 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T08:41:16.127 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T08:41:16.127 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.num_levels: 7 2024-09-16T08:41:16.127 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T08:41:16.128 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T08:41:16.128 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T08:41:16.128 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T08:41:16.128 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T08:41:16.128 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T08:41:16.128 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T08:41:16.128 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:41:16.128 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T08:41:16.128 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T08:41:16.128 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:41:16.129 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:41:16.129 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T08:41:16.129 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T08:41:16.130 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T08:41:16.130 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T08:41:16.130 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:41:16.130 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:41:16.130 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T08:41:16.130 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T08:41:16.130 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:41:16.130 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T08:41:16.130 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T08:41:16.130 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T08:41:16.130 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T08:41:16.130 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T08:41:16.131 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T08:41:16.131 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T08:41:16.131 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T08:41:16.131 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T08:41:16.131 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T08:41:16.131 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T08:41:16.131 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T08:41:16.131 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T08:41:16.131 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T08:41:16.131 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T08:41:16.131 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T08:41:16.131 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T08:41:16.131 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T08:41:16.131 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T08:41:16.132 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T08:41:16.132 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T08:41:16.132 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T08:41:16.132 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T08:41:16.132 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T08:41:16.132 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T08:41:16.132 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T08:41:16.132 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T08:41:16.132 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T08:41:16.132 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T08:41:16.132 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T08:41:16.132 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T08:41:16.132 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T08:41:16.132 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T08:41:16.133 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.inplace_update_support: 0 2024-09-16T08:41:16.133 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T08:41:16.133 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T08:41:16.133 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T08:41:16.133 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T08:41:16.133 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.bloom_locality: 0 2024-09-16T08:41:16.133 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.max_successive_merges: 0 2024-09-16T08:41:16.133 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T08:41:16.133 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T08:41:16.133 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T08:41:16.133 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T08:41:16.133 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.ttl: 2592000 2024-09-16T08:41:16.133 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T08:41:16.134 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T08:41:16.134 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T08:41:16.134 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.enable_blob_files: false 2024-09-16T08:41:16.134 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.min_blob_size: 0 2024-09-16T08:41:16.134 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T08:41:16.134 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T08:41:16.134 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T08:41:16.134 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T08:41:16.134 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T08:41:16.134 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T08:41:16.134 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T08:41:16.134 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T08:41:16.134 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T08:41:16.134 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T08:41:16.135 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 824714b9-3022-444a-945e-1e4cbcc95ca2 2024-09-16T08:41:16.135 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726476075715284, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T08:41:16.135 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T08:41:16.135 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726476075715833, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1613, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 488, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 366, "raw_average_value_size": 73, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726476075, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "824714b9-3022-444a-945e-1e4cbcc95ca2", "db_session_id": "2FKTWNAOG9VORB4DA7A5", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T08:41:16.135 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726476075715963, "job": 1, "event": "recovery_finished"} 2024-09-16T08:41:16.135 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T08:41:16.135 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T08:41:16.135 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55abc8244e00 2024-09-16T08:41:16.135 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: DB pointer 0x55abc8346000 2024-09-16T08:41:16.135 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T08:41:16.135 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T08:41:16.135 INFO:journalctl@ceph.mon.b.smithi169.stdout: ** DB Stats ** 2024-09-16T08:41:16.135 INFO:journalctl@ceph.mon.b.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:41:16.135 INFO:journalctl@ceph.mon.b.smithi169.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T08:41:16.136 INFO:journalctl@ceph.mon.b.smithi169.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:41:16.136 INFO:journalctl@ceph.mon.b.smithi169.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:41:16.136 INFO:journalctl@ceph.mon.b.smithi169.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T08:41:16.136 INFO:journalctl@ceph.mon.b.smithi169.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:41:16.136 INFO:journalctl@ceph.mon.b.smithi169.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:41:16.136 INFO:journalctl@ceph.mon.b.smithi169.stdout: 2024-09-16T08:41:16.136 INFO:journalctl@ceph.mon.b.smithi169.stdout: ** Compaction Stats [default] ** 2024-09-16T08:41:16.136 INFO:journalctl@ceph.mon.b.smithi169.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:41:16.136 INFO:journalctl@ceph.mon.b.smithi169.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T08:41:16.136 INFO:journalctl@ceph.mon.b.smithi169.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T08:41:16.136 INFO:journalctl@ceph.mon.b.smithi169.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T08:41:16.136 INFO:journalctl@ceph.mon.b.smithi169.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T08:41:16.136 INFO:journalctl@ceph.mon.b.smithi169.stdout: 2024-09-16T08:41:16.136 INFO:journalctl@ceph.mon.b.smithi169.stdout: ** Compaction Stats [default] ** 2024-09-16T08:41:16.137 INFO:journalctl@ceph.mon.b.smithi169.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:41:16.137 INFO:journalctl@ceph.mon.b.smithi169.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T08:41:16.137 INFO:journalctl@ceph.mon.b.smithi169.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T08:41:16.137 INFO:journalctl@ceph.mon.b.smithi169.stdout: 2024-09-16T08:41:16.137 INFO:journalctl@ceph.mon.b.smithi169.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T08:41:16.137 INFO:journalctl@ceph.mon.b.smithi169.stdout: 2024-09-16T08:41:16.137 INFO:journalctl@ceph.mon.b.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:41:16.137 INFO:journalctl@ceph.mon.b.smithi169.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T08:41:16.137 INFO:journalctl@ceph.mon.b.smithi169.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T08:41:16.137 INFO:journalctl@ceph.mon.b.smithi169.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T08:41:16.137 INFO:journalctl@ceph.mon.b.smithi169.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T08:41:16.137 INFO:journalctl@ceph.mon.b.smithi169.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T08:41:16.137 INFO:journalctl@ceph.mon.b.smithi169.stdout: Cumulative compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:41:16.138 INFO:journalctl@ceph.mon.b.smithi169.stdout: Interval compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:41:16.138 INFO:journalctl@ceph.mon.b.smithi169.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T08:41:16.138 INFO:journalctl@ceph.mon.b.smithi169.stdout: Block cache BinnedLRUCache@0x55abc8215610#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T08:41:16.138 INFO:journalctl@ceph.mon.b.smithi169.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T08:41:16.138 INFO:journalctl@ceph.mon.b.smithi169.stdout: 2024-09-16T08:41:16.138 INFO:journalctl@ceph.mon.b.smithi169.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T08:41:16.138 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-16T08:41:16.138 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: using public_addrv v1:172.21.15.169:6789/0 2024-09-16T08:41:16.138 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: starting mon.b rank -1 at public addrs v1:172.21.15.169:6789/0 at bind addrs v1:172.21.15.169:6789/0 mon_data /var/lib/ceph/mon/ceph-b fsid 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:41:16.138 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: mon.b@-1(???) e0 preinit fsid 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:41:16.138 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: mon.b@-1(synchronizing).mds e1 new map 2024-09-16T08:41:16.138 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-16T08:41:16.138 INFO:journalctl@ceph.mon.b.smithi169.stdout: e1 2024-09-16T08:41:16.139 INFO:journalctl@ceph.mon.b.smithi169.stdout: btime 2024-09-16T08:39:54:060588+0000 2024-09-16T08:41:16.139 INFO:journalctl@ceph.mon.b.smithi169.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:41:16.139 INFO:journalctl@ceph.mon.b.smithi169.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T08:41:16.139 INFO:journalctl@ceph.mon.b.smithi169.stdout: legacy client fscid: -1 2024-09-16T08:41:16.139 INFO:journalctl@ceph.mon.b.smithi169.stdout: 2024-09-16T08:41:16.139 INFO:journalctl@ceph.mon.b.smithi169.stdout: No filesystems configured 2024-09-16T08:41:16.139 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T08:41:16.139 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T08:41:16.139 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T08:41:16.139 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T08:41:16.139 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T08:41:16.139 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T08:41:16.140 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T08:41:16.140 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:41:16.140 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:41:16.140 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:41:16.140 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:16.140 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.169:0/2599051096' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T08:41:16.140 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:16.140 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 ceph-mon[33465]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-16T08:41:16.140 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 bash[33427]: 4b357747a229cec57221a812a464350639522901b2c397f7c81db0008a17f211 2024-09-16T08:41:16.140 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:15 smithi169 systemd[1]: Started Ceph mon.b for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:41:20.757 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.755+0000 7f7844dbe640 1 -- 172.21.15.169:0/2927658365 >> v1:172.21.15.160:6789/0 conn(0x7f7820005770 legacy=0x7f7820007c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:20.758 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.756+0000 7f7844dbe640 1 -- 172.21.15.169:0/2927658365 shutdown_connections 2024-09-16T08:41:20.758 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.756+0000 7f7844dbe640 1 -- 172.21.15.169:0/2927658365 >> 172.21.15.169:0/2927658365 conn(0x7f78400fd180 msgr2=0x7f78400ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:20.758 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.756+0000 7f7844dbe640 1 -- 172.21.15.169:0/2927658365 shutdown_connections 2024-09-16T08:41:20.758 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.756+0000 7f7844dbe640 1 -- 172.21.15.169:0/2927658365 wait complete. 2024-09-16T08:41:20.758 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.757+0000 7f7844dbe640 1 Processor -- start 2024-09-16T08:41:20.758 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.757+0000 7f7844dbe640 1 -- start start 2024-09-16T08:41:20.759 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.758+0000 7f7844dbe640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f78401a5630 con 0x7f7820005770 2024-09-16T08:41:20.759 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.758+0000 7f7844dbe640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f78401a6810 con 0x7f7840199bd0 2024-09-16T08:41:20.759 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.758+0000 7f7835d9b640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f7840199bd0 0x7f78401a3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:34322/0 (socket says 172.21.15.169:34322) 2024-09-16T08:41:20.759 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.758+0000 7f7835d9b640 1 -- 172.21.15.169:0/4023465096 learned_addr learned my addr 172.21.15.169:0/4023465096 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:41:20.759 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.758+0000 7f7837fff640 1 -- 172.21.15.169:0/4023465096 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 531735982 0 0) 0x7f78401a5630 con 0x7f7820005770 2024-09-16T08:41:20.759 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.758+0000 7f7837fff640 1 -- 172.21.15.169:0/4023465096 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f782c003520 con 0x7f7820005770 2024-09-16T08:41:20.760 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.759+0000 7f7837fff640 1 -- 172.21.15.169:0/4023465096 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1411483874 0 0) 0x7f782c003520 con 0x7f7820005770 2024-09-16T08:41:20.760 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.759+0000 7f7837fff640 1 -- 172.21.15.169:0/4023465096 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f78401a5630 con 0x7f7820005770 2024-09-16T08:41:20.760 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.759+0000 7f7837fff640 1 -- 172.21.15.169:0/4023465096 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 239+0+0 (unknown 923198682 0 0) 0x7f7830001aa0 con 0x7f7820005770 2024-09-16T08:41:20.761 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.759+0000 7f7837fff640 1 -- 172.21.15.169:0/4023465096 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3140052692 0 0) 0x7f78401a5630 con 0x7f7820005770 2024-09-16T08:41:20.761 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.759+0000 7f7837fff640 1 -- 172.21.15.169:0/4023465096 >> v1:172.21.15.169:6789/0 conn(0x7f7840199bd0 legacy=0x7f78401a3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:20.761 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.759+0000 7f7837fff640 1 -- 172.21.15.169:0/4023465096 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78401a79f0 con 0x7f7820005770 2024-09-16T08:41:20.761 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.759+0000 7f7844dbe640 1 -- 172.21.15.169:0/4023465096 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f78401a5860 con 0x7f7820005770 2024-09-16T08:41:20.761 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.759+0000 7f7844dbe640 1 -- 172.21.15.169:0/4023465096 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f78401a5e40 con 0x7f7820005770 2024-09-16T08:41:20.761 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.759+0000 7f7837fff640 1 -- 172.21.15.169:0/4023465096 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f7830001d10 con 0x7f7820005770 2024-09-16T08:41:20.761 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.759+0000 7f7837fff640 1 -- 172.21.15.169:0/4023465096 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 239+0+0 (unknown 923198682 0 0) 0x7f7830004a80 con 0x7f7820005770 2024-09-16T08:41:20.762 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.760+0000 7f7844dbe640 1 -- 172.21.15.169:0/4023465096 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7820008d30 con 0x7f7820005770 2024-09-16T08:41:20.762 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.761+0000 7f7837fff640 1 -- 172.21.15.169:0/4023465096 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 13) ==== 51787+0+0 (unknown 1985690445 0 0) 0x7f78300116a0 con 0x7f7820005770 2024-09-16T08:41:20.762 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.762+0000 7f7837fff640 1 -- 172.21.15.169:0/4023465096 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 3038969552 0 0) 0x7f78300011c0 con 0x7f7820005770 2024-09-16T08:41:20.767 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:20.767+0000 7f7837fff640 1 -- 172.21.15.169:0/4023465096 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f78300179d0 con 0x7f7820005770 2024-09-16T08:41:21.019 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:21.018+0000 7f7844dbe640 1 -- 172.21.15.169:0/4023465096 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7820009410 con 0x7f7820005770 2024-09-16T08:41:21.019 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:21.018+0000 7f7837fff640 1 -- 172.21.15.169:0/4023465096 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+915 (unknown 3557084514 0 2243734004) 0x7f7830004450 con 0x7f7820005770 2024-09-16T08:41:21.019 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:41:21.019 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":2,"fsid":"27cca2fc-7407-11ef-bceb-c7b262605968","modified":"2024-09-16T08:41:15.739632Z","created":"2024-09-16T08:39:50.898512Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T08:41:21.020 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 2 2024-09-16T08:41:21.020 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:41:21.020 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: mon.a calling monitor election 2024-09-16T08:41:21.020 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:21.020 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:21.020 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:21.020 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:21.020 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: mon.b calling monitor election 2024-09-16T08:41:21.020 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:21.021 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:21.021 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:21.021 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:21.021 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:21.021 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T08:41:21.021 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: monmap epoch 2 2024-09-16T08:41:21.021 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: fsid 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:41:21.021 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: last_changed 2024-09-16T08:41:15.739632+0000 2024-09-16T08:41:21.021 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: created 2024-09-16T08:39:50.898512+0000 2024-09-16T08:41:21.021 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: min_mon_release 19 (squid) 2024-09-16T08:41:21.021 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: election_strategy: 1 2024-09-16T08:41:21.021 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: 0: v1:172.21.15.160:6789/0 mon.a 2024-09-16T08:41:21.021 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: 1: v1:172.21.15.169:6789/0 mon.b 2024-09-16T08:41:21.022 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: fsmap 2024-09-16T08:41:21.022 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T08:41:21.022 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: mgrmap e13: y(active, since 34s) 2024-09-16T08:41:21.022 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: overall HEALTH_OK 2024-09-16T08:41:21.022 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:21.022 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:21.022 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:21.022 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:41:21.022 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:20 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:21.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:21.020+0000 7f7844dbe640 1 -- 172.21.15.169:0/4023465096 >> v1:172.21.15.160:6800/1286008384 conn(0x7f782c0409a0 legacy=0x7f782c042e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:21.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:21.020+0000 7f7844dbe640 1 -- 172.21.15.169:0/4023465096 >> v1:172.21.15.160:6789/0 conn(0x7f7820005770 legacy=0x7f78401994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:21.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:21.021+0000 7f7844dbe640 1 -- 172.21.15.169:0/4023465096 shutdown_connections 2024-09-16T08:41:21.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:21.021+0000 7f7844dbe640 1 -- 172.21.15.169:0/4023465096 >> 172.21.15.169:0/4023465096 conn(0x7f78400fd180 msgr2=0x7f7840104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:21.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:21.021+0000 7f7844dbe640 1 -- 172.21.15.169:0/4023465096 shutdown_connections 2024-09-16T08:41:21.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:21.021+0000 7f7844dbe640 1 -- 172.21.15.169:0/4023465096 wait complete. 2024-09-16T08:41:21.160 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:41:21.160 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: mon.a calling monitor election 2024-09-16T08:41:21.160 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:21.160 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:21.160 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:21.160 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:21.160 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: mon.b calling monitor election 2024-09-16T08:41:21.161 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:21.161 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:21.161 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:21.161 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:21.161 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:21.161 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T08:41:21.161 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: monmap epoch 2 2024-09-16T08:41:21.161 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: fsid 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:41:21.161 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: last_changed 2024-09-16T08:41:15.739632+0000 2024-09-16T08:41:21.161 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: created 2024-09-16T08:39:50.898512+0000 2024-09-16T08:41:21.161 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: min_mon_release 19 (squid) 2024-09-16T08:41:21.161 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: election_strategy: 1 2024-09-16T08:41:21.162 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: 0: v1:172.21.15.160:6789/0 mon.a 2024-09-16T08:41:21.162 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: 1: v1:172.21.15.169:6789/0 mon.b 2024-09-16T08:41:21.162 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: fsmap 2024-09-16T08:41:21.162 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T08:41:21.162 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: mgrmap e13: y(active, since 34s) 2024-09-16T08:41:21.162 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: overall HEALTH_OK 2024-09-16T08:41:21.162 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:21.162 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:21.162 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:21.162 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:41:21.162 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:20 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:22.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:21 smithi169 ceph-mon[33465]: Deploying daemon mon.c on smithi160 2024-09-16T08:41:22.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:21 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.169:0/4023465096' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T08:41:22.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:21 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:22.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:21 smithi160 ceph-mon[30137]: Deploying daemon mon.c on smithi160 2024-09-16T08:41:22.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:21 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.169:0/4023465096' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T08:41:22.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:21 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:22.632 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T08:41:22.633 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:41:22.887 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:41:22.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:22 smithi160 ceph-mon[30137]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:23.014 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:22 smithi169 ceph-mon[33465]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:23.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.314+0000 7ff99c387640 1 -- 172.21.15.169:0/1100627821 >> v1:172.21.15.160:6789/0 conn(0x7ff994101730 legacy=0x7ff994101b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:23.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.314+0000 7ff99c387640 1 -- 172.21.15.169:0/1100627821 shutdown_connections 2024-09-16T08:41:23.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.314+0000 7ff99c387640 1 -- 172.21.15.169:0/1100627821 >> 172.21.15.169:0/1100627821 conn(0x7ff9940fd180 msgr2=0x7ff9940ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:23.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.314+0000 7ff99c387640 1 -- 172.21.15.169:0/1100627821 shutdown_connections 2024-09-16T08:41:23.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.314+0000 7ff99c387640 1 -- 172.21.15.169:0/1100627821 wait complete. 2024-09-16T08:41:23.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.315+0000 7ff99c387640 1 Processor -- start 2024-09-16T08:41:23.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.315+0000 7ff99c387640 1 -- start start 2024-09-16T08:41:23.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.315+0000 7ff99c387640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff994198c60 con 0x7ff9941987d0 2024-09-16T08:41:23.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.315+0000 7ff99c387640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff994198e30 con 0x7ff994101730 2024-09-16T08:41:23.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.315+0000 7ff999922640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7ff9941987d0 0x7ff9941a2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.169:45104/0 (socket says 172.21.15.169:45104) 2024-09-16T08:41:23.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.315+0000 7ff999922640 1 -- 172.21.15.169:0/1701476450 learned_addr learned my addr 172.21.15.169:0/1701476450 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:41:23.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 297539762 0 0) 0x7ff994198e30 con 0x7ff994101730 2024-09-16T08:41:23.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff980003520 con 0x7ff994101730 2024-09-16T08:41:23.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 854155783 0 0) 0x7ff994198c60 con 0x7ff9941987d0 2024-09-16T08:41:23.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff994198e30 con 0x7ff9941987d0 2024-09-16T08:41:23.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 567723790 0 0) 0x7ff994198e30 con 0x7ff9941987d0 2024-09-16T08:41:23.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff994198c60 con 0x7ff9941987d0 2024-09-16T08:41:23.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 239+0+0 (unknown 923198682 0 0) 0x7ff9840017c0 con 0x7ff9941987d0 2024-09-16T08:41:23.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3739063732 0 0) 0x7ff980003520 con 0x7ff994101730 2024-09-16T08:41:23.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff994198e30 con 0x7ff994101730 2024-09-16T08:41:23.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 239+0+0 (unknown 923198682 0 0) 0x7ff990001640 con 0x7ff994101730 2024-09-16T08:41:23.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3641106232 0 0) 0x7ff994198c60 con 0x7ff9941987d0 2024-09-16T08:41:23.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 >> v1:172.21.15.169:6789/0 conn(0x7ff994101730 legacy=0x7ff9941980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:23.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9941a5880 con 0x7ff9941987d0 2024-09-16T08:41:23.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff99c387640 1 -- 172.21.15.169:0/1701476450 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff9941a3860 con 0x7ff9941987d0 2024-09-16T08:41:23.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff99c387640 1 -- 172.21.15.169:0/1701476450 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff9941a3e40 con 0x7ff9941987d0 2024-09-16T08:41:23.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7ff984004d10 con 0x7ff9941987d0 2024-09-16T08:41:23.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.316+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 239+0+0 (unknown 923198682 0 0) 0x7ff984005b50 con 0x7ff9941987d0 2024-09-16T08:41:23.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.317+0000 7ff99c387640 1 -- 172.21.15.169:0/1701476450 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9780050b0 con 0x7ff9941987d0 2024-09-16T08:41:23.320 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.317+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 13) ==== 51787+0+0 (unknown 1985690445 0 0) 0x7ff984012770 con 0x7ff9941987d0 2024-09-16T08:41:23.320 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.318+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 3038969552 0 0) 0x7ff98404f3f0 con 0x7ff9941987d0 2024-09-16T08:41:23.322 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.322+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7ff984018ac0 con 0x7ff9941987d0 2024-09-16T08:41:23.568 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.567+0000 7ff99c387640 1 -- 172.21.15.169:0/1701476450 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff9780053a0 con 0x7ff9941987d0 2024-09-16T08:41:23.569 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.567+0000 7ff98b7fe640 1 -- 172.21.15.169:0/1701476450 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+915 (unknown 3557084514 0 2243734004) 0x7ff984021d20 con 0x7ff9941987d0 2024-09-16T08:41:23.569 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:41:23.569 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":2,"fsid":"27cca2fc-7407-11ef-bceb-c7b262605968","modified":"2024-09-16T08:41:15.739632Z","created":"2024-09-16T08:39:50.898512Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T08:41:23.570 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 2 2024-09-16T08:41:23.571 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.570+0000 7ff99c387640 1 -- 172.21.15.169:0/1701476450 >> v1:172.21.15.160:6800/1286008384 conn(0x7ff980040c20 legacy=0x7ff9800430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:23.571 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.570+0000 7ff99c387640 1 -- 172.21.15.169:0/1701476450 >> v1:172.21.15.160:6789/0 conn(0x7ff9941987d0 legacy=0x7ff9941a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:23.571 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.570+0000 7ff99c387640 1 -- 172.21.15.169:0/1701476450 shutdown_connections 2024-09-16T08:41:23.571 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.570+0000 7ff99c387640 1 -- 172.21.15.169:0/1701476450 >> 172.21.15.169:0/1701476450 conn(0x7ff9940fd180 msgr2=0x7ff994107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:23.571 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.570+0000 7ff99c387640 1 -- 172.21.15.169:0/1701476450 shutdown_connections 2024-09-16T08:41:23.571 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:23.570+0000 7ff99c387640 1 -- 172.21.15.169:0/1701476450 wait complete. 2024-09-16T08:41:23.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:23 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.169:0/1701476450' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T08:41:24.068 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:23 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.169:0/1701476450' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T08:41:24.068 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 podman[39990]: 2024-09-16 08:41:24.068168669 +0000 UTC m=+0.118896260 container create 9e91ddc50d3df5817dbe9fdb9d73061eba4f4dcdae79fefb045f0e1d605c2c2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-16T08:41:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 podman[39990]: 2024-09-16 08:41:24.328991453 +0000 UTC m=+0.379719002 container init 9e91ddc50d3df5817dbe9fdb9d73061eba4f4dcdae79fefb045f0e1d605c2c2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T08:41:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 podman[39990]: 2024-09-16 08:41:24.334019073 +0000 UTC m=+0.384746622 container start 9e91ddc50d3df5817dbe9fdb9d73061eba4f4dcdae79fefb045f0e1d605c2c2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-c, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T08:41:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T08:41:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev), process ceph-mon, pid 7 2024-09-16T08:41:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: pidfile_write: ignore empty --pid-file 2024-09-16T08:41:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: load: jerasure load: lrc 2024-09-16T08:41:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: RocksDB version: 7.9.2 2024-09-16T08:41:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Git sha 0 2024-09-16T08:41:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Compile date 2024-09-13 14:50:41 2024-09-16T08:41:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: DB SUMMARY 2024-09-16T08:41:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: DB Session ID: US59MXLWT07FWK0UG5JF 2024-09-16T08:41:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: CURRENT file: CURRENT 2024-09-16T08:41:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T08:41:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-16T08:41:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-16T08:41:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 566 ; 2024-09-16T08:41:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.error_if_exists: 0 2024-09-16T08:41:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.create_if_missing: 0 2024-09-16T08:41:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.paranoid_checks: 1 2024-09-16T08:41:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T08:41:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T08:41:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T08:41:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.env: 0x56219589c120 2024-09-16T08:41:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T08:41:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.info_log: 0x5621977816e0 2024-09-16T08:41:24.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T08:41:24.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.statistics: (nil) 2024-09-16T08:41:24.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.use_fsync: 0 2024-09-16T08:41:24.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_log_file_size: 0 2024-09-16T08:41:24.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T08:41:24.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T08:41:24.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T08:41:24.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T08:41:24.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.allow_fallocate: 1 2024-09-16T08:41:24.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T08:41:24.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T08:41:24.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.use_direct_reads: 0 2024-09-16T08:41:24.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T08:41:24.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T08:41:24.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.db_log_dir: 2024-09-16T08:41:24.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.wal_dir: 2024-09-16T08:41:24.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T08:41:24.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T08:41:24.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T08:41:24.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T08:41:24.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T08:41:24.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T08:41:24.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T08:41:24.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T08:41:24.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.write_buffer_manager: 0x562197785b80 2024-09-16T08:41:24.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T08:41:24.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T08:41:24.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T08:41:24.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T08:41:24.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T08:41:24.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T08:41:24.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T08:41:24.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T08:41:24.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.unordered_write: 0 2024-09-16T08:41:24.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T08:41:24.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T08:41:24.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T08:41:24.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T08:41:24.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.row_cache: None 2024-09-16T08:41:24.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.wal_filter: None 2024-09-16T08:41:24.705 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T08:41:24.705 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T08:41:24.705 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.two_write_queues: 0 2024-09-16T08:41:24.705 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T08:41:24.705 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.wal_compression: 0 2024-09-16T08:41:24.706 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.atomic_flush: 0 2024-09-16T08:41:24.706 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T08:41:24.706 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T08:41:24.706 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T08:41:24.706 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.log_readahead_size: 0 2024-09-16T08:41:24.706 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T08:41:24.706 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T08:41:24.706 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T08:41:24.706 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T08:41:24.706 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T08:41:24.706 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T08:41:24.706 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T08:41:24.706 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_background_jobs: 2 2024-09-16T08:41:24.707 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_background_compactions: -1 2024-09-16T08:41:24.707 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_subcompactions: 1 2024-09-16T08:41:24.707 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T08:41:24.707 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T08:41:24.707 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T08:41:24.707 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T08:41:24.707 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T08:41:24.707 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T08:41:24.707 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T08:41:24.707 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T08:41:24.707 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_open_files: -1 2024-09-16T08:41:24.707 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T08:41:24.707 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T08:41:24.707 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T08:41:24.708 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T08:41:24.708 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_background_flushes: -1 2024-09-16T08:41:24.708 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Compression algorithms supported: 2024-09-16T08:41:24.708 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: kZSTD supported: 0 2024-09-16T08:41:24.708 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: kXpressCompression supported: 0 2024-09-16T08:41:24.708 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: kBZip2Compression supported: 0 2024-09-16T08:41:24.708 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T08:41:24.708 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: kLZ4Compression supported: 1 2024-09-16T08:41:24.708 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: kZlibCompression supported: 1 2024-09-16T08:41:24.708 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T08:41:24.708 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: kSnappyCompression supported: 1 2024-09-16T08:41:24.708 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T08:41:24.708 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T08:41:24.708 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-16T08:41:24.708 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T08:41:24.709 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T08:41:24.709 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.merge_operator: 2024-09-16T08:41:24.709 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compaction_filter: None 2024-09-16T08:41:24.709 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T08:41:24.709 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T08:41:24.709 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T08:41:24.709 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T08:41:24.709 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562197781300) 2024-09-16T08:41:24.709 INFO:journalctl@ceph.mon.c.smithi160.stdout: cache_index_and_filter_blocks: 1 2024-09-16T08:41:24.709 INFO:journalctl@ceph.mon.c.smithi160.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T08:41:24.709 INFO:journalctl@ceph.mon.c.smithi160.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T08:41:24.710 INFO:journalctl@ceph.mon.c.smithi160.stdout: pin_top_level_index_and_filter: 1 2024-09-16T08:41:24.710 INFO:journalctl@ceph.mon.c.smithi160.stdout: index_type: 0 2024-09-16T08:41:24.710 INFO:journalctl@ceph.mon.c.smithi160.stdout: data_block_index_type: 0 2024-09-16T08:41:24.710 INFO:journalctl@ceph.mon.c.smithi160.stdout: index_shortening: 1 2024-09-16T08:41:24.710 INFO:journalctl@ceph.mon.c.smithi160.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T08:41:24.710 INFO:journalctl@ceph.mon.c.smithi160.stdout: checksum: 4 2024-09-16T08:41:24.710 INFO:journalctl@ceph.mon.c.smithi160.stdout: no_block_cache: 0 2024-09-16T08:41:24.710 INFO:journalctl@ceph.mon.c.smithi160.stdout: block_cache: 0x562197775610 2024-09-16T08:41:24.710 INFO:journalctl@ceph.mon.c.smithi160.stdout: block_cache_name: BinnedLRUCache 2024-09-16T08:41:24.710 INFO:journalctl@ceph.mon.c.smithi160.stdout: block_cache_options: 2024-09-16T08:41:24.711 INFO:journalctl@ceph.mon.c.smithi160.stdout: capacity : 536870912 2024-09-16T08:41:24.711 INFO:journalctl@ceph.mon.c.smithi160.stdout: num_shard_bits : 4 2024-09-16T08:41:24.711 INFO:journalctl@ceph.mon.c.smithi160.stdout: strict_capacity_limit : 0 2024-09-16T08:41:24.711 INFO:journalctl@ceph.mon.c.smithi160.stdout: high_pri_pool_ratio: 0.000 2024-09-16T08:41:24.711 INFO:journalctl@ceph.mon.c.smithi160.stdout: block_cache_compressed: (nil) 2024-09-16T08:41:24.711 INFO:journalctl@ceph.mon.c.smithi160.stdout: persistent_cache: (nil) 2024-09-16T08:41:24.711 INFO:journalctl@ceph.mon.c.smithi160.stdout: block_size: 4096 2024-09-16T08:41:24.711 INFO:journalctl@ceph.mon.c.smithi160.stdout: block_size_deviation: 10 2024-09-16T08:41:24.712 INFO:journalctl@ceph.mon.c.smithi160.stdout: block_restart_interval: 16 2024-09-16T08:41:24.712 INFO:journalctl@ceph.mon.c.smithi160.stdout: index_block_restart_interval: 1 2024-09-16T08:41:24.712 INFO:journalctl@ceph.mon.c.smithi160.stdout: metadata_block_size: 4096 2024-09-16T08:41:24.712 INFO:journalctl@ceph.mon.c.smithi160.stdout: partition_filters: 0 2024-09-16T08:41:24.712 INFO:journalctl@ceph.mon.c.smithi160.stdout: use_delta_encoding: 1 2024-09-16T08:41:24.712 INFO:journalctl@ceph.mon.c.smithi160.stdout: filter_policy: bloomfilter 2024-09-16T08:41:24.712 INFO:journalctl@ceph.mon.c.smithi160.stdout: whole_key_filtering: 1 2024-09-16T08:41:24.712 INFO:journalctl@ceph.mon.c.smithi160.stdout: verify_compression: 0 2024-09-16T08:41:24.712 INFO:journalctl@ceph.mon.c.smithi160.stdout: read_amp_bytes_per_bit: 0 2024-09-16T08:41:24.712 INFO:journalctl@ceph.mon.c.smithi160.stdout: format_version: 5 2024-09-16T08:41:24.712 INFO:journalctl@ceph.mon.c.smithi160.stdout: enable_index_compression: 1 2024-09-16T08:41:24.712 INFO:journalctl@ceph.mon.c.smithi160.stdout: block_align: 0 2024-09-16T08:41:24.712 INFO:journalctl@ceph.mon.c.smithi160.stdout: max_auto_readahead_size: 262144 2024-09-16T08:41:24.713 INFO:journalctl@ceph.mon.c.smithi160.stdout: prepopulate_block_cache: 0 2024-09-16T08:41:24.713 INFO:journalctl@ceph.mon.c.smithi160.stdout: initial_auto_readahead_size: 8192 2024-09-16T08:41:24.713 INFO:journalctl@ceph.mon.c.smithi160.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T08:41:24.713 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T08:41:24.713 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T08:41:24.713 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compression: NoCompression 2024-09-16T08:41:24.713 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T08:41:24.713 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T08:41:24.713 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T08:41:24.713 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.num_levels: 7 2024-09-16T08:41:24.713 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T08:41:24.713 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T08:41:24.714 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T08:41:24.714 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T08:41:24.714 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T08:41:24.714 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T08:41:24.714 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T08:41:24.714 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:41:24.714 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T08:41:24.714 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T08:41:24.714 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:41:24.714 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:41:24.715 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T08:41:24.715 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T08:41:24.715 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T08:41:24.715 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T08:41:24.715 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:41:24.715 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:41:24.715 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T08:41:24.715 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T08:41:24.715 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:41:24.715 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T08:41:24.715 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T08:41:24.715 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T08:41:24.715 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T08:41:24.715 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T08:41:24.716 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T08:41:24.716 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T08:41:24.716 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T08:41:24.716 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T08:41:24.716 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T08:41:24.716 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T08:41:24.716 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T08:41:24.716 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T08:41:24.716 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T08:41:24.716 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T08:41:24.716 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T08:41:24.716 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T08:41:24.717 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T08:41:24.717 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T08:41:24.717 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T08:41:24.717 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T08:41:24.717 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T08:41:24.717 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T08:41:24.717 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T08:41:24.717 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T08:41:24.717 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T08:41:24.717 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T08:41:24.717 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T08:41:24.718 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T08:41:24.718 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T08:41:24.718 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T08:41:24.718 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T08:41:24.718 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T08:41:24.718 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.inplace_update_support: 0 2024-09-16T08:41:24.718 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T08:41:24.718 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T08:41:24.718 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T08:41:24.718 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T08:41:24.718 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.bloom_locality: 0 2024-09-16T08:41:24.718 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.max_successive_merges: 0 2024-09-16T08:41:24.718 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T08:41:24.719 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T08:41:24.719 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T08:41:24.719 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T08:41:24.719 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.ttl: 2592000 2024-09-16T08:41:24.719 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T08:41:24.719 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T08:41:24.719 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T08:41:24.719 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.enable_blob_files: false 2024-09-16T08:41:24.719 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.min_blob_size: 0 2024-09-16T08:41:24.719 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T08:41:24.719 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T08:41:24.720 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T08:41:24.720 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T08:41:24.720 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T08:41:24.720 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T08:41:24.720 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T08:41:24.720 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T08:41:24.720 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T08:41:24.720 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T08:41:24.721 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bf56c542-48a9-4177-acb0-a39598a5dcd3 2024-09-16T08:41:24.721 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726476084360573, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T08:41:24.721 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T08:41:24.721 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726476084360959, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1703, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 578, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 456, "raw_average_value_size": 91, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726476084, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bf56c542-48a9-4177-acb0-a39598a5dcd3", "db_session_id": "US59MXLWT07FWK0UG5JF", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T08:41:24.721 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726476084361050, "job": 1, "event": "recovery_finished"} 2024-09-16T08:41:24.721 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T08:41:24.721 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T08:41:24.721 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5621977a4e00 2024-09-16T08:41:24.721 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: DB pointer 0x5621978a6000 2024-09-16T08:41:24.721 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T08:41:24.721 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T08:41:24.721 INFO:journalctl@ceph.mon.c.smithi160.stdout: ** DB Stats ** 2024-09-16T08:41:24.721 INFO:journalctl@ceph.mon.c.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:41:24.722 INFO:journalctl@ceph.mon.c.smithi160.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T08:41:24.722 INFO:journalctl@ceph.mon.c.smithi160.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:41:24.722 INFO:journalctl@ceph.mon.c.smithi160.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:41:24.722 INFO:journalctl@ceph.mon.c.smithi160.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T08:41:24.722 INFO:journalctl@ceph.mon.c.smithi160.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:41:24.722 INFO:journalctl@ceph.mon.c.smithi160.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:41:24.722 INFO:journalctl@ceph.mon.c.smithi160.stdout: 2024-09-16T08:41:24.722 INFO:journalctl@ceph.mon.c.smithi160.stdout: ** Compaction Stats [default] ** 2024-09-16T08:41:24.722 INFO:journalctl@ceph.mon.c.smithi160.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:41:24.722 INFO:journalctl@ceph.mon.c.smithi160.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T08:41:24.722 INFO:journalctl@ceph.mon.c.smithi160.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T08:41:24.723 INFO:journalctl@ceph.mon.c.smithi160.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T08:41:24.723 INFO:journalctl@ceph.mon.c.smithi160.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T08:41:24.723 INFO:journalctl@ceph.mon.c.smithi160.stdout: 2024-09-16T08:41:24.723 INFO:journalctl@ceph.mon.c.smithi160.stdout: ** Compaction Stats [default] ** 2024-09-16T08:41:24.723 INFO:journalctl@ceph.mon.c.smithi160.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:41:24.723 INFO:journalctl@ceph.mon.c.smithi160.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T08:41:24.723 INFO:journalctl@ceph.mon.c.smithi160.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T08:41:24.723 INFO:journalctl@ceph.mon.c.smithi160.stdout: 2024-09-16T08:41:24.723 INFO:journalctl@ceph.mon.c.smithi160.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T08:41:24.723 INFO:journalctl@ceph.mon.c.smithi160.stdout: 2024-09-16T08:41:24.723 INFO:journalctl@ceph.mon.c.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:41:24.724 INFO:journalctl@ceph.mon.c.smithi160.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T08:41:24.724 INFO:journalctl@ceph.mon.c.smithi160.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T08:41:24.724 INFO:journalctl@ceph.mon.c.smithi160.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T08:41:24.724 INFO:journalctl@ceph.mon.c.smithi160.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T08:41:24.724 INFO:journalctl@ceph.mon.c.smithi160.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T08:41:24.724 INFO:journalctl@ceph.mon.c.smithi160.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:41:24.724 INFO:journalctl@ceph.mon.c.smithi160.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:41:24.724 INFO:journalctl@ceph.mon.c.smithi160.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T08:41:24.724 INFO:journalctl@ceph.mon.c.smithi160.stdout: Block cache BinnedLRUCache@0x562197775610#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T08:41:24.724 INFO:journalctl@ceph.mon.c.smithi160.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T08:41:24.724 INFO:journalctl@ceph.mon.c.smithi160.stdout: 2024-09-16T08:41:24.724 INFO:journalctl@ceph.mon.c.smithi160.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T08:41:24.724 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-16T08:41:24.724 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: using public_addrv v1:172.21.15.160:6790/0 2024-09-16T08:41:24.725 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: starting mon.c rank -1 at public addrs v1:172.21.15.160:6790/0 at bind addrs v1:172.21.15.160:6790/0 mon_data /var/lib/ceph/mon/ceph-c fsid 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:41:24.725 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.c@-1(???) e0 preinit fsid 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:41:24.725 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.c@-1(synchronizing).mds e1 new map 2024-09-16T08:41:24.725 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-16T08:41:24.725 INFO:journalctl@ceph.mon.c.smithi160.stdout: e1 2024-09-16T08:41:24.725 INFO:journalctl@ceph.mon.c.smithi160.stdout: btime 2024-09-16T08:39:54:060588+0000 2024-09-16T08:41:24.725 INFO:journalctl@ceph.mon.c.smithi160.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:41:24.725 INFO:journalctl@ceph.mon.c.smithi160.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T08:41:24.725 INFO:journalctl@ceph.mon.c.smithi160.stdout: legacy client fscid: -1 2024-09-16T08:41:24.725 INFO:journalctl@ceph.mon.c.smithi160.stdout: 2024-09-16T08:41:24.725 INFO:journalctl@ceph.mon.c.smithi160.stdout: No filesystems configured 2024-09-16T08:41:24.725 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T08:41:24.725 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T08:41:24.725 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T08:41:24.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T08:41:24.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T08:41:24.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T08:41:24.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T08:41:24.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:41:24.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:41:24.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:41:24.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:24.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.169:0/2599051096' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T08:41:24.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:24.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:41:24.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.a calling monitor election 2024-09-16T08:41:24.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:24.727 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:24.727 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:24.727 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:24.727 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.b calling monitor election 2024-09-16T08:41:24.727 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:24.727 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:24.727 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:24.727 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:24.727 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:24.728 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T08:41:24.728 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: monmap epoch 2 2024-09-16T08:41:24.728 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: fsid 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:41:24.728 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: last_changed 2024-09-16T08:41:15.739632+0000 2024-09-16T08:41:24.728 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: created 2024-09-16T08:39:50.898512+0000 2024-09-16T08:41:24.728 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: min_mon_release 19 (squid) 2024-09-16T08:41:24.728 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: election_strategy: 1 2024-09-16T08:41:24.728 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: 0: v1:172.21.15.160:6789/0 mon.a 2024-09-16T08:41:24.728 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: 1: v1:172.21.15.169:6789/0 mon.b 2024-09-16T08:41:24.729 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: fsmap 2024-09-16T08:41:24.729 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T08:41:24.729 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mgrmap e13: y(active, since 34s) 2024-09-16T08:41:24.729 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: overall HEALTH_OK 2024-09-16T08:41:24.729 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:24.729 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:24.729 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:24.729 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:41:24.729 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:24.729 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: Deploying daemon mon.c on smithi160 2024-09-16T08:41:24.729 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.169:0/4023465096' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T08:41:24.730 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:24.730 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:24.730 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.169:0/1701476450' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T08:41:24.730 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 ceph-mon[40028]: mon.c@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-09-16T08:41:24.730 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 bash[39990]: 9e91ddc50d3df5817dbe9fdb9d73061eba4f4dcdae79fefb045f0e1d605c2c2f 2024-09-16T08:41:24.730 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:24 smithi160 systemd[1]: Started Ceph mon.c for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:41:25.131 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T08:41:25.132 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:41:25.384 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:41:29.396 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.394+0000 7f2293cda640 1 -- 172.21.15.169:0/1636985464 >> v1:172.21.15.160:6789/0 conn(0x7f2270003560 legacy=0x7f22700059f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:29.396 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.394+0000 7f2293cda640 1 -- 172.21.15.169:0/1636985464 shutdown_connections 2024-09-16T08:41:29.397 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.394+0000 7f2293cda640 1 -- 172.21.15.169:0/1636985464 >> 172.21.15.169:0/1636985464 conn(0x7f228c0fd180 msgr2=0x7f228c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:29.397 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.394+0000 7f2293cda640 1 -- 172.21.15.169:0/1636985464 shutdown_connections 2024-09-16T08:41:29.397 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.394+0000 7f2293cda640 1 -- 172.21.15.169:0/1636985464 wait complete. 2024-09-16T08:41:29.397 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.395+0000 7f2293cda640 1 Processor -- start 2024-09-16T08:41:29.397 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.395+0000 7f2293cda640 1 -- start start 2024-09-16T08:41:29.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.395+0000 7f2293cda640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f228c1a9400 con 0x7f228c1a5730 2024-09-16T08:41:29.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.395+0000 7f2293cda640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f228c1aa600 con 0x7f228c199d70 2024-09-16T08:41:29.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.395+0000 7f2293cda640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f228c1ab800 con 0x7f2270003560 2024-09-16T08:41:29.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.395+0000 7f2291275640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f228c199d70 0x7f228c1a4010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:51306/0 (socket says 172.21.15.169:51306) 2024-09-16T08:41:29.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.395+0000 7f2291275640 1 -- 172.21.15.169:0/3050874261 learned_addr learned my addr 172.21.15.169:0/3050874261 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:41:29.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.396+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 156842282 0 0) 0x7f228c1aa600 con 0x7f228c199d70 2024-09-16T08:41:29.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.396+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2278003c00 con 0x7f228c199d70 2024-09-16T08:41:29.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.396+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 980211745 0 0) 0x7f228c1a9400 con 0x7f228c1a5730 2024-09-16T08:41:29.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.396+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f228c1aa600 con 0x7f228c1a5730 2024-09-16T08:41:29.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.396+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 81664114 0 0) 0x7f2278003c00 con 0x7f228c199d70 2024-09-16T08:41:29.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.396+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f228c1a9400 con 0x7f228c199d70 2024-09-16T08:41:29.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.396+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f227c0018b0 con 0x7f228c199d70 2024-09-16T08:41:29.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.396+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 <== mon.1 v1:172.21.15.169:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3837803991 0 0) 0x7f228c1a9400 con 0x7f228c199d70 2024-09-16T08:41:29.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.396+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 >> v1:172.21.15.160:6790/0 conn(0x7f2270003560 legacy=0x7f228c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:29.400 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.396+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 >> v1:172.21.15.160:6789/0 conn(0x7f228c1a5730 legacy=0x7f228c1a7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:29.400 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.396+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228c1acaa0 con 0x7f228c199d70 2024-09-16T08:41:29.400 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.396+0000 7f2293cda640 1 -- 172.21.15.169:0/3050874261 --> v1:172.21.15.169:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f228c1a9630 con 0x7f228c199d70 2024-09-16T08:41:29.400 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.397+0000 7f2293cda640 1 -- 172.21.15.169:0/3050874261 --> v1:172.21.15.169:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f228c1a9c60 con 0x7f228c199d70 2024-09-16T08:41:29.400 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.397+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 <== mon.1 v1:172.21.15.169:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f227c004d10 con 0x7f228c199d70 2024-09-16T08:41:29.400 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.397+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 <== mon.1 v1:172.21.15.169:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f227c005c00 con 0x7f228c199d70 2024-09-16T08:41:29.400 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.398+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 <== mon.1 v1:172.21.15.169:6789/0 7 ==== mgrmap(e 13) ==== 51787+0+0 (unknown 1985690445 0 0) 0x7f227c012820 con 0x7f228c199d70 2024-09-16T08:41:29.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.398+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 <== mon.1 v1:172.21.15.169:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 3038969552 0 0) 0x7f227c050330 con 0x7f228c199d70 2024-09-16T08:41:29.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.398+0000 7f2293cda640 1 -- 172.21.15.169:0/3050874261 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2270006aa0 con 0x7f228c199d70 2024-09-16T08:41:29.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.404+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 <== mon.1 v1:172.21.15.169:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f227c018b50 con 0x7f228c199d70 2024-09-16T08:41:29.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.662+0000 7f2293cda640 1 -- 172.21.15.169:0/3050874261 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2270006d90 con 0x7f228c199d70 2024-09-16T08:41:29.664 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:29.664 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:41:29.664 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: mon.a calling monitor election 2024-09-16T08:41:29.664 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: mon.b calling monitor election 2024-09-16T08:41:29.664 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:29.665 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.665 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.665 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:29.665 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.665 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: mon.c calling monitor election 2024-09-16T08:41:29.665 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.665 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:29.665 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.665 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.665 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T08:41:29.665 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: monmap epoch 3 2024-09-16T08:41:29.665 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: fsid 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:41:29.665 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: last_changed 2024-09-16T08:41:24.380243+0000 2024-09-16T08:41:29.666 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: created 2024-09-16T08:39:50.898512+0000 2024-09-16T08:41:29.666 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: min_mon_release 19 (squid) 2024-09-16T08:41:29.666 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: election_strategy: 1 2024-09-16T08:41:29.666 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: 0: v1:172.21.15.160:6789/0 mon.a 2024-09-16T08:41:29.666 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: 1: v1:172.21.15.169:6789/0 mon.b 2024-09-16T08:41:29.666 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: 2: v1:172.21.15.160:6790/0 mon.c 2024-09-16T08:41:29.666 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: fsmap 2024-09-16T08:41:29.666 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T08:41:29.666 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: mgrmap e13: y(active, since 43s) 2024-09-16T08:41:29.666 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: overall HEALTH_OK 2024-09-16T08:41:29.666 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:29.666 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:29.666 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:29.666 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:29 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:29.667 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:41:29.667 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":3,"fsid":"27cca2fc-7407-11ef-bceb-c7b262605968","modified":"2024-09-16T08:41:24.380243Z","created":"2024-09-16T08:39:50.898512Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-16T08:41:29.668 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.663+0000 7f2282ffd640 1 -- 172.21.15.169:0/3050874261 <== mon.1 v1:172.21.15.169:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1133 (unknown 409899127 0 1308320859) 0x7f227c021db0 con 0x7f228c199d70 2024-09-16T08:41:29.668 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 3 2024-09-16T08:41:29.668 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.666+0000 7f2293cda640 1 -- 172.21.15.169:0/3050874261 >> v1:172.21.15.160:6800/1286008384 conn(0x7f2278040a00 legacy=0x7f2278042ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:29.668 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.666+0000 7f2293cda640 1 -- 172.21.15.169:0/3050874261 >> v1:172.21.15.169:6789/0 conn(0x7f228c199d70 legacy=0x7f228c1a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:29.668 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.666+0000 7f2293cda640 1 -- 172.21.15.169:0/3050874261 shutdown_connections 2024-09-16T08:41:29.668 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.666+0000 7f2293cda640 1 -- 172.21.15.169:0/3050874261 >> 172.21.15.169:0/3050874261 conn(0x7f228c0fd180 msgr2=0x7f228c0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:29.668 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.666+0000 7f2293cda640 1 -- 172.21.15.169:0/3050874261 shutdown_connections 2024-09-16T08:41:29.668 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:29.666+0000 7f2293cda640 1 -- 172.21.15.169:0/3050874261 wait complete. 2024-09-16T08:41:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:41:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: mon.a calling monitor election 2024-09-16T08:41:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: mon.b calling monitor election 2024-09-16T08:41:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:29.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: mon.c calling monitor election 2024-09-16T08:41:29.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:29.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T08:41:29.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: monmap epoch 3 2024-09-16T08:41:29.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: fsid 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:41:29.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: last_changed 2024-09-16T08:41:24.380243+0000 2024-09-16T08:41:29.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: created 2024-09-16T08:39:50.898512+0000 2024-09-16T08:41:29.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: min_mon_release 19 (squid) 2024-09-16T08:41:29.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: election_strategy: 1 2024-09-16T08:41:29.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: 0: v1:172.21.15.160:6789/0 mon.a 2024-09-16T08:41:29.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: 1: v1:172.21.15.169:6789/0 mon.b 2024-09-16T08:41:29.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: 2: v1:172.21.15.160:6790/0 mon.c 2024-09-16T08:41:29.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: fsmap 2024-09-16T08:41:29.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T08:41:29.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: mgrmap e13: y(active, since 43s) 2024-09-16T08:41:29.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: overall HEALTH_OK 2024-09-16T08:41:29.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:29.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:29.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:29.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:29.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:29.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:41:29.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: mon.a calling monitor election 2024-09-16T08:41:29.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: mon.b calling monitor election 2024-09-16T08:41:29.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:41:29.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:29.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: mon.c calling monitor election 2024-09-16T08:41:29.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:29.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:29.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T08:41:29.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: monmap epoch 3 2024-09-16T08:41:29.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: fsid 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:41:29.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: last_changed 2024-09-16T08:41:24.380243+0000 2024-09-16T08:41:29.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: created 2024-09-16T08:39:50.898512+0000 2024-09-16T08:41:29.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: min_mon_release 19 (squid) 2024-09-16T08:41:29.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: election_strategy: 1 2024-09-16T08:41:29.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: 0: v1:172.21.15.160:6789/0 mon.a 2024-09-16T08:41:29.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: 1: v1:172.21.15.169:6789/0 mon.b 2024-09-16T08:41:29.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: 2: v1:172.21.15.160:6790/0 mon.c 2024-09-16T08:41:29.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: fsmap 2024-09-16T08:41:29.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T08:41:29.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: mgrmap e13: y(active, since 43s) 2024-09-16T08:41:29.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: overall HEALTH_OK 2024-09-16T08:41:29.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:29.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:29.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:29.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:29 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:30.279 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-16T08:41:30.280 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-16T08:41:30.548 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:41:30.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:30.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.169:0/3050874261' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T08:41:30.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:30.670 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:30.670 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.169:0/3050874261' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T08:41:30.670 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:30.671 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:30.671 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.169:0/3050874261' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T08:41:30.671 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:41:31.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.010+0000 7f3940d06640 1 -- 172.21.15.160:0/1427634589 >> v1:172.21.15.160:6789/0 conn(0x7f393c101590 legacy=0x7f393c101970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:31.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.011+0000 7f3940d06640 1 -- 172.21.15.160:0/1427634589 shutdown_connections 2024-09-16T08:41:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.011+0000 7f3940d06640 1 -- 172.21.15.160:0/1427634589 >> 172.21.15.160:0/1427634589 conn(0x7f393c078170 msgr2=0x7f393c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.011+0000 7f3940d06640 1 -- 172.21.15.160:0/1427634589 shutdown_connections 2024-09-16T08:41:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.011+0000 7f3940d06640 1 -- 172.21.15.160:0/1427634589 wait complete. 2024-09-16T08:41:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.011+0000 7f3940d06640 1 Processor -- start 2024-09-16T08:41:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.011+0000 7f3940d06640 1 -- start start 2024-09-16T08:41:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f3940d06640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f393c1a7100 con 0x7f393c198c90 2024-09-16T08:41:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f3940d06640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f393c1a8300 con 0x7f393c101590 2024-09-16T08:41:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f3940d06640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f393c1a9500 con 0x7f393c198800 2024-09-16T08:41:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f393ad9d640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f393c198c90 0x7f393c1a58a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:34398/0 (socket says 172.21.15.160:34398) 2024-09-16T08:41:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f393ad9d640 1 -- 172.21.15.160:0/2285968238 learned_addr learned my addr 172.21.15.160:0/2285968238 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:41:31.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2796405223 0 0) 0x7f393c1a9500 con 0x7f393c198800 2024-09-16T08:41:31.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3928003c00 con 0x7f393c198800 2024-09-16T08:41:31.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 641165319 0 0) 0x7f393c1a7100 con 0x7f393c198c90 2024-09-16T08:41:31.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f393c1a9500 con 0x7f393c198c90 2024-09-16T08:41:31.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1894679477 0 0) 0x7f3928003c00 con 0x7f393c198800 2024-09-16T08:41:31.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f393c1a7100 con 0x7f393c198800 2024-09-16T08:41:31.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 250889377 0 0) 0x7f393c1a8300 con 0x7f393c101590 2024-09-16T08:41:31.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3928003c00 con 0x7f393c101590 2024-09-16T08:41:31.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f392c0018b0 con 0x7f393c198800 2024-09-16T08:41:31.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3754014273 0 0) 0x7f393c1a9500 con 0x7f393c198c90 2024-09-16T08:41:31.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f393c1a8300 con 0x7f393c198c90 2024-09-16T08:41:31.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f3934001b30 con 0x7f393c198c90 2024-09-16T08:41:31.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2028381920 0 0) 0x7f393c1a7100 con 0x7f393c198800 2024-09-16T08:41:31.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.012+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 >> v1:172.21.15.169:6789/0 conn(0x7f393c101590 legacy=0x7f393c1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:31.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.013+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 >> v1:172.21.15.160:6789/0 conn(0x7f393c198c90 legacy=0x7f393c1a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:31.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.013+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393c1aa7a0 con 0x7f393c198800 2024-09-16T08:41:31.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.013+0000 7f3940d06640 1 -- 172.21.15.160:0/2285968238 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f393c1a7330 con 0x7f393c198800 2024-09-16T08:41:31.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.013+0000 7f3940d06640 1 -- 172.21.15.160:0/2285968238 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f393c1a7960 con 0x7f393c198800 2024-09-16T08:41:31.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.013+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f392c003a60 con 0x7f393c198800 2024-09-16T08:41:31.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.013+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f392c004980 con 0x7f393c198800 2024-09-16T08:41:31.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.013+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 13) ==== 51787+0+0 (unknown 1985690445 0 0) 0x7f392c011580 con 0x7f393c198800 2024-09-16T08:41:31.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.014+0000 7f3940d06640 1 -- 172.21.15.160:0/2285968238 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391c0050b0 con 0x7f393c198800 2024-09-16T08:41:31.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.014+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 3038969552 0 0) 0x7f392c04e1b0 con 0x7f393c198800 2024-09-16T08:41:31.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.019+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f392c0178b0 con 0x7f393c198800 2024-09-16T08:41:31.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.195+0000 7f3940d06640 1 -- 172.21.15.160:0/2285968238 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f391c0053a0 con 0x7f393c198800 2024-09-16T08:41:31.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.196+0000 7f39337fe640 1 -- 172.21.15.160:0/2285968238 <== mon.2 v1:172.21.15.160:6790/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+193 (unknown 2189217731 0 2174481370) 0x7f392c020b10 con 0x7f393c198800 2024-09-16T08:41:31.197 INFO:teuthology.orchestra.run.smithi160.stdout:# minimal ceph.conf for 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:41:31.197 INFO:teuthology.orchestra.run.smithi160.stdout:[global] 2024-09-16T08:41:31.197 INFO:teuthology.orchestra.run.smithi160.stdout: fsid = 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:41:31.197 INFO:teuthology.orchestra.run.smithi160.stdout: mon_host = 172.21.15.160:6789/0 172.21.15.169:6789/0 v1:172.21.15.160:6790/0 2024-09-16T08:41:31.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.199+0000 7f3940d06640 1 -- 172.21.15.160:0/2285968238 >> v1:172.21.15.160:6800/1286008384 conn(0x7f3928040b10 legacy=0x7f3928042fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:31.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.199+0000 7f3940d06640 1 -- 172.21.15.160:0/2285968238 >> v1:172.21.15.160:6790/0 conn(0x7f393c198800 legacy=0x7f393c1a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:31.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.199+0000 7f3940d06640 1 -- 172.21.15.160:0/2285968238 shutdown_connections 2024-09-16T08:41:31.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.199+0000 7f3940d06640 1 -- 172.21.15.160:0/2285968238 >> 172.21.15.160:0/2285968238 conn(0x7f393c078170 msgr2=0x7f393c107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:31.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.199+0000 7f3940d06640 1 -- 172.21.15.160:0/2285968238 shutdown_connections 2024-09-16T08:41:31.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:31.199+0000 7f3940d06640 1 -- 172.21.15.160:0/2285968238 wait complete. 2024-09-16T08:41:31.841 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[30137]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:31.841 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:31.841 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:31.841 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:31.841 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2285968238' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:31.841 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:31.841 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:31.841 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:31.841 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:41:31.842 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[40028]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:31.842 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:31.842 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:31.842 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:31.842 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2285968238' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:31.842 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:31.842 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:31.842 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:31.842 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:31 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:41:31.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:31 smithi169 ceph-mon[33465]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:31.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:31 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:31.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:31 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:31.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:31 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2285968238' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:31 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:31 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:31 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:31 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:41:31.872 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-16T08:41:31.872 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:41:31.872 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T08:41:31.901 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:41:31.902 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:41:31.967 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T08:41:31.967 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T08:41:31.995 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T08:41:31.995 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:41:32.061 INFO:tasks.cephadm:Adding mgr.y on smithi160 2024-09-16T08:41:32.061 INFO:tasks.cephadm:Adding mgr.x on smithi169 2024-09-16T08:41:32.062 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch apply mgr '2;smithi160=y;smithi169=x' 2024-09-16T08:41:32.342 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:41:32.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: Updating smithi160:/etc/ceph/ceph.conf 2024-09-16T08:41:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T08:41:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: Updating smithi169:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:41:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: Updating smithi160:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:41:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T08:41:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:41:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T08:41:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[40028]: Reconfiguring daemon mon.a on smithi160 2024-09-16T08:41:32.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: Updating smithi160:/etc/ceph/ceph.conf 2024-09-16T08:41:32.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T08:41:32.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: Updating smithi169:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:41:32.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: Updating smithi160:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:41:32.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T08:41:32.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:41:32.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T08:41:32.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:32.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:32 smithi160 ceph-mon[30137]: Reconfiguring daemon mon.a on smithi160 2024-09-16T08:41:32.783 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: Updating smithi160:/etc/ceph/ceph.conf 2024-09-16T08:41:32.783 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T08:41:32.783 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: Updating smithi169:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:41:32.784 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: Updating smithi160:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:41:32.784 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.784 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.784 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.784 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.784 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.784 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.784 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.784 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.784 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:32.784 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T08:41:32.784 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:41:32.785 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T08:41:32.785 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:32.785 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:32 smithi169 ceph-mon[33465]: Reconfiguring daemon mon.a on smithi160 2024-09-16T08:41:32.787 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.781+0000 7f657114e640 1 -- 172.21.15.169:0/4217398803 >> v1:172.21.15.160:6789/0 conn(0x7f656c103ab0 legacy=0x7f656c103eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:32.787 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.781+0000 7f657114e640 1 -- 172.21.15.169:0/4217398803 shutdown_connections 2024-09-16T08:41:32.787 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.781+0000 7f657114e640 1 -- 172.21.15.169:0/4217398803 >> 172.21.15.169:0/4217398803 conn(0x7f656c0fc3a0 msgr2=0x7f656c0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:32.787 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.781+0000 7f657114e640 1 -- 172.21.15.169:0/4217398803 shutdown_connections 2024-09-16T08:41:32.787 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.781+0000 7f657114e640 1 -- 172.21.15.169:0/4217398803 wait complete. 2024-09-16T08:41:32.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.782+0000 7f657114e640 1 Processor -- start 2024-09-16T08:41:32.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.782+0000 7f657114e640 1 -- start start 2024-09-16T08:41:32.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.782+0000 7f657114e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f656c19ade0 con 0x7f656c198c40 2024-09-16T08:41:32.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.782+0000 7f657114e640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f656c1a9fa0 con 0x7f656c103ab0 2024-09-16T08:41:32.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.782+0000 7f657114e640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f656c1ab180 con 0x7f656c19a810 2024-09-16T08:41:32.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.782+0000 7f656ad9d640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f656c103ab0 0x7f656c102eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:51308/0 (socket says 172.21.15.169:51308) 2024-09-16T08:41:32.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.782+0000 7f656ad9d640 1 -- 172.21.15.169:0/2836653865 learned_addr learned my addr 172.21.15.169:0/2836653865 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:41:32.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.782+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2813322274 0 0) 0x7f656c1a9fa0 con 0x7f656c103ab0 2024-09-16T08:41:32.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.782+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6554003c50 con 0x7f656c103ab0 2024-09-16T08:41:32.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.782+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1933374899 0 0) 0x7f656c19ade0 con 0x7f656c198c40 2024-09-16T08:41:32.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f656c1a9fa0 con 0x7f656c198c40 2024-09-16T08:41:32.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1142514676 0 0) 0x7f656c1ab180 con 0x7f656c19a810 2024-09-16T08:41:32.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f656c19ade0 con 0x7f656c19a810 2024-09-16T08:41:32.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 440152019 0 0) 0x7f656c1a9fa0 con 0x7f656c198c40 2024-09-16T08:41:32.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f656c1ab180 con 0x7f656c198c40 2024-09-16T08:41:32.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f6560001890 con 0x7f656c198c40 2024-09-16T08:41:32.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3491825048 0 0) 0x7f6554003c50 con 0x7f656c103ab0 2024-09-16T08:41:32.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f656c1a9fa0 con 0x7f656c103ab0 2024-09-16T08:41:32.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f655c0013b0 con 0x7f656c103ab0 2024-09-16T08:41:32.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 199829222 0 0) 0x7f656c19ade0 con 0x7f656c19a810 2024-09-16T08:41:32.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6554003c50 con 0x7f656c19a810 2024-09-16T08:41:32.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f6564001b70 con 0x7f656c19a810 2024-09-16T08:41:32.790 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 196954403 0 0) 0x7f656c1ab180 con 0x7f656c198c40 2024-09-16T08:41:32.790 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 >> v1:172.21.15.160:6790/0 conn(0x7f656c19a810 legacy=0x7f656c1a7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:32.790 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 >> v1:172.21.15.169:6789/0 conn(0x7f656c103ab0 legacy=0x7f656c102eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:32.790 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656c1ac400 con 0x7f656c198c40 2024-09-16T08:41:32.790 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f657114e640 1 -- 172.21.15.169:0/2836653865 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f656c1a8ff0 con 0x7f656c198c40 2024-09-16T08:41:32.790 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f657114e640 1 -- 172.21.15.169:0/2836653865 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f656c1a9620 con 0x7f656c198c40 2024-09-16T08:41:32.790 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f6560003a80 con 0x7f656c198c40 2024-09-16T08:41:32.790 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.783+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f65600049f0 con 0x7f656c198c40 2024-09-16T08:41:32.790 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.784+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 13) ==== 51787+0+0 (unknown 1985690445 0 0) 0x7f6560011450 con 0x7f656c198c40 2024-09-16T08:41:32.790 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.784+0000 7f657114e640 1 -- 172.21.15.169:0/2836653865 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65580050b0 con 0x7f656c198c40 2024-09-16T08:41:32.790 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.784+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 3038969552 0 0) 0x7f656004ef30 con 0x7f656c198c40 2024-09-16T08:41:32.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.789+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f65600176f0 con 0x7f656c198c40 2024-09-16T08:41:32.976 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.974+0000 7f657114e640 1 -- 172.21.15.169:0/2836653865 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi160=y;smithi169=x", "target": ["mon-mgr", ""]}) -- 0x7f6558000bc0 con 0x7f6554040ce0 2024-09-16T08:41:32.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.981+0000 7f6553fff640 1 -- 172.21.15.169:0/2836653865 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (unknown 0 0 325935098) 0x7f6558000bc0 con 0x7f6554040ce0 2024-09-16T08:41:32.982 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mgr update... 2024-09-16T08:41:32.985 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.983+0000 7f657114e640 1 -- 172.21.15.169:0/2836653865 >> v1:172.21.15.160:6800/1286008384 conn(0x7f6554040ce0 legacy=0x7f65540431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:32.985 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.983+0000 7f657114e640 1 -- 172.21.15.169:0/2836653865 >> v1:172.21.15.160:6789/0 conn(0x7f656c198c40 legacy=0x7f656c1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:32.985 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.983+0000 7f657114e640 1 -- 172.21.15.169:0/2836653865 shutdown_connections 2024-09-16T08:41:32.985 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.983+0000 7f657114e640 1 -- 172.21.15.169:0/2836653865 >> 172.21.15.169:0/2836653865 conn(0x7f656c0fc3a0 msgr2=0x7f656c0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:32.985 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.983+0000 7f657114e640 1 -- 172.21.15.169:0/2836653865 shutdown_connections 2024-09-16T08:41:32.985 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:41:32.984+0000 7f657114e640 1 -- 172.21.15.169:0/2836653865 wait complete. 2024-09-16T08:41:33.606 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:33 smithi169 ceph-mon[33465]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:33.606 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:33.606 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:33.606 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:33.606 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-16T08:41:33.606 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-16T08:41:33.607 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:33.608 DEBUG:teuthology.orchestra.run.smithi169:mgr.x> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mgr.x.service 2024-09-16T08:41:33.610 DEBUG:tasks.cephadm:set 0 configs 2024-09-16T08:41:33.610 INFO:tasks.cephadm:Deploying OSDs... 2024-09-16T08:41:33.610 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:41:33.610 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T08:41:33.626 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T08:41:33.627 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_1 2024-09-16T08:41:33.683 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T08:41:33.683 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:41:33.683 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 968 Links: 1 2024-09-16T08:41:33.683 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:41:33.683 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:41:33.683 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-09-16 08:41:23.792340937 +0000 2024-09-16T08:41:33.683 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-09-16 08:36:35.278568005 +0000 2024-09-16T08:41:33.683 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-09-16 08:36:35.278568005 +0000 2024-09-16T08:41:33.683 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-09-16 08:36:35.278568005 +0000 2024-09-16T08:41:33.684 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T08:41:33.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:33 smithi160 ceph-mon[30137]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:33.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:33.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:33.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:33.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-16T08:41:33.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-16T08:41:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:33 smithi160 ceph-mon[40028]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-16T08:41:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-16T08:41:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:33.721 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-09-16T08:41:33.722 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-09-16T08:41:33.722 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000246386 s, 2.1 MB/s 2024-09-16T08:41:33.723 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T08:41:33.779 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_2 2024-09-16T08:41:33.834 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T08:41:33.834 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:41:33.834 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 984 Links: 1 2024-09-16T08:41:33.834 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:41:33.834 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:41:33.834 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-09-16 08:41:23.792340937 +0000 2024-09-16T08:41:33.834 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-09-16 08:36:35.279567991 +0000 2024-09-16T08:41:33.834 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-09-16 08:36:35.279567991 +0000 2024-09-16T08:41:33.834 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-09-16 08:36:35.279567991 +0000 2024-09-16T08:41:33.835 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T08:41:33.897 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-09-16T08:41:33.897 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-09-16T08:41:33.897 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.00020805 s, 2.5 MB/s 2024-09-16T08:41:33.898 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T08:41:33.955 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_3 2024-09-16T08:41:34.010 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T08:41:34.010 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:41:34.010 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 975 Links: 1 2024-09-16T08:41:34.011 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:41:34.011 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:41:34.011 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-09-16 08:41:23.792340937 +0000 2024-09-16T08:41:34.011 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-09-16 08:36:35.279567991 +0000 2024-09-16T08:41:34.011 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-09-16 08:36:35.279567991 +0000 2024-09-16T08:41:34.011 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-09-16 08:36:35.279567991 +0000 2024-09-16T08:41:34.011 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T08:41:34.074 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-09-16T08:41:34.074 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-09-16T08:41:34.074 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000201302 s, 2.5 MB/s 2024-09-16T08:41:34.075 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T08:41:34.132 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_4 2024-09-16T08:41:34.187 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T08:41:34.188 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:41:34.188 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-16T08:41:34.188 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:41:34.188 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:41:34.188 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-09-16 08:41:23.793340922 +0000 2024-09-16T08:41:34.188 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-09-16 08:36:35.279567991 +0000 2024-09-16T08:41:34.188 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-09-16 08:36:35.279567991 +0000 2024-09-16T08:41:34.188 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-09-16 08:36:35.279567991 +0000 2024-09-16T08:41:34.188 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T08:41:34.251 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-09-16T08:41:34.251 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-09-16T08:41:34.251 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000231779 s, 2.2 MB/s 2024-09-16T08:41:34.253 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T08:41:34.309 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T08:41:34.310 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T08:41:34.328 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T08:41:34.329 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2024-09-16T08:41:34.387 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T08:41:34.387 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:41:34.387 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 981 Links: 1 2024-09-16T08:41:34.387 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:41:34.387 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:41:34.387 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-09-16 08:41:15.167184509 +0000 2024-09-16T08:41:34.387 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-09-16 08:36:34.024451327 +0000 2024-09-16T08:41:34.387 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-09-16 08:36:34.024451327 +0000 2024-09-16T08:41:34.387 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-09-16 08:36:34.024451327 +0000 2024-09-16T08:41:34.388 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T08:41:34.453 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-09-16T08:41:34.453 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-09-16T08:41:34.453 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.00022866 s, 2.2 MB/s 2024-09-16T08:41:34.454 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T08:41:34.512 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2024-09-16T08:41:34.568 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T08:41:34.569 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:41:34.569 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 976 Links: 1 2024-09-16T08:41:34.569 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:41:34.569 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:41:34.569 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-09-16 08:41:15.167184509 +0000 2024-09-16T08:41:34.569 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-09-16 08:36:34.023451540 +0000 2024-09-16T08:41:34.569 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-09-16 08:36:34.023451540 +0000 2024-09-16T08:41:34.569 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-09-16 08:36:34.023451540 +0000 2024-09-16T08:41:34.569 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T08:41:34.634 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-09-16T08:41:34.634 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-09-16T08:41:34.634 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000323617 s, 1.6 MB/s 2024-09-16T08:41:34.636 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T08:41:34.693 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2024-09-16T08:41:34.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[40028]: from='client.14211 v1:172.21.15.169:0/2836653865' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi160=y;smithi169=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:41:34.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[40028]: Saving service mgr spec with placement smithi160=y;smithi169=x;count:2 2024-09-16T08:41:34.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[40028]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T08:41:34.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[40028]: Reconfiguring daemon mgr.y on smithi160 2024-09-16T08:41:34.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:34.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:41:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T08:41:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[30137]: from='client.14211 v1:172.21.15.169:0/2836653865' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi160=y;smithi169=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:41:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[30137]: Saving service mgr spec with placement smithi160=y;smithi169=x;count:2 2024-09-16T08:41:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[30137]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T08:41:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[30137]: Reconfiguring daemon mgr.y on smithi160 2024-09-16T08:41:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:41:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T08:41:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:34 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:34.749 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T08:41:34.749 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:41:34.749 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-09-16T08:41:34.749 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:41:34.749 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:41:34.750 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-09-16 08:41:15.168184487 +0000 2024-09-16T08:41:34.750 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-09-16 08:36:34.022451753 +0000 2024-09-16T08:41:34.750 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-09-16 08:36:34.022451753 +0000 2024-09-16T08:41:34.750 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-09-16 08:36:34.022451753 +0000 2024-09-16T08:41:34.750 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T08:41:34.810 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:34 smithi169 ceph-mon[33465]: from='client.14211 v1:172.21.15.169:0/2836653865' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi160=y;smithi169=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:41:34.810 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:34 smithi169 ceph-mon[33465]: Saving service mgr spec with placement smithi160=y;smithi169=x;count:2 2024-09-16T08:41:34.811 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:34 smithi169 ceph-mon[33465]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T08:41:34.811 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:34 smithi169 ceph-mon[33465]: Reconfiguring daemon mgr.y on smithi160 2024-09-16T08:41:34.811 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:34 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:34.811 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:34 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:34.811 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:34 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:41:34.811 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:34 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T08:41:34.811 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:34 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:34.812 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-09-16T08:41:34.813 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-09-16T08:41:34.813 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000194568 s, 2.6 MB/s 2024-09-16T08:41:34.814 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T08:41:34.870 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2024-09-16T08:41:34.925 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T08:41:34.926 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:41:34.926 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-16T08:41:34.926 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:41:34.926 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:41:34.926 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-09-16 08:41:15.168184487 +0000 2024-09-16T08:41:34.926 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-09-16 08:36:34.022451753 +0000 2024-09-16T08:41:34.926 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-09-16 08:36:34.022451753 +0000 2024-09-16T08:41:34.926 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-09-16 08:36:34.022451753 +0000 2024-09-16T08:41:34.926 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T08:41:34.988 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-09-16T08:41:34.988 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-09-16T08:41:34.988 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000204668 s, 2.5 MB/s 2024-09-16T08:41:34.990 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T08:41:35.046 INFO:tasks.cephadm:Deploying osd.0 on smithi160 with /dev/vg_nvme/lv_4... 2024-09-16T08:41:35.046 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T08:41:35.528 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:41:35.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:35 smithi169 ceph-mon[33465]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:35.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:35 smithi169 ceph-mon[33465]: Reconfiguring mon.c (monmap changed)... 2024-09-16T08:41:35.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:35 smithi169 ceph-mon[33465]: Reconfiguring daemon mon.c on smithi160 2024-09-16T08:41:35.901 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:35 smithi160 ceph-mon[30137]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:35.902 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:35 smithi160 ceph-mon[30137]: Reconfiguring mon.c (monmap changed)... 2024-09-16T08:41:35.902 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:35 smithi160 ceph-mon[30137]: Reconfiguring daemon mon.c on smithi160 2024-09-16T08:41:35.902 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:35 smithi160 ceph-mon[40028]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:35.902 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:35 smithi160 ceph-mon[40028]: Reconfiguring mon.c (monmap changed)... 2024-09-16T08:41:35.902 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:35 smithi160 ceph-mon[40028]: Reconfiguring daemon mon.c on smithi160 2024-09-16T08:41:37.145 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:36 smithi160 ceph-mon[30137]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:37.145 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:36 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:37.145 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:36 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:37.145 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:36 smithi160 ceph-mon[30137]: Reconfiguring mon.b (monmap changed)... 2024-09-16T08:41:37.146 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:36 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:41:37.146 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:36 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T08:41:37.146 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:36 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:37.146 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:36 smithi160 ceph-mon[30137]: Reconfiguring daemon mon.b on smithi169 2024-09-16T08:41:37.146 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:36 smithi160 ceph-mon[40028]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:37.146 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:36 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:37.146 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:36 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:37.146 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:36 smithi160 ceph-mon[40028]: Reconfiguring mon.b (monmap changed)... 2024-09-16T08:41:37.147 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:36 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:41:37.147 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:36 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T08:41:37.147 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:36 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:37.147 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:36 smithi160 ceph-mon[40028]: Reconfiguring daemon mon.b on smithi169 2024-09-16T08:41:37.203 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:36 smithi169 ceph-mon[33465]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:37.203 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:36 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:37.203 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:36 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:37.203 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:36 smithi169 ceph-mon[33465]: Reconfiguring mon.b (monmap changed)... 2024-09-16T08:41:37.203 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:36 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:41:37.203 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:36 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T08:41:37.203 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:36 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:37.203 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:36 smithi169 ceph-mon[33465]: Reconfiguring daemon mon.b on smithi169 2024-09-16T08:41:38.564 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:38.564 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:38.564 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:38.564 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:38.564 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:41:38.564 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:38.564 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-16T08:41:38.564 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T08:41:38.564 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-16T08:41:38.565 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:38.565 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[30137]: Deploying daemon mgr.x on smithi169 2024-09-16T08:41:38.565 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:38.565 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:38.565 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:38.565 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:38.565 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:41:38.565 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:38.565 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-16T08:41:38.565 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T08:41:38.565 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-16T08:41:38.566 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:38.566 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:38 smithi160 ceph-mon[40028]: Deploying daemon mgr.x on smithi169 2024-09-16T08:41:38.566 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:41:38.591 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_4 2024-09-16T08:41:38.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:38 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:38.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:38 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:38.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:38 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:38.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:38 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:38.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:38 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:41:38.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:38 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:38.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:38 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-16T08:41:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:38 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T08:41:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:38 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-16T08:41:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:38 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:38 smithi169 ceph-mon[33465]: Deploying daemon mgr.x on smithi169 2024-09-16T08:41:38.837 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:41:39.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.251+0000 7f09b0810640 1 -- 172.21.15.160:0/591365433 >> v1:172.21.15.160:6789/0 conn(0x7f09ac105b60 legacy=0x7f09ac107f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:39.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.251+0000 7f09b0810640 1 -- 172.21.15.160:0/591365433 shutdown_connections 2024-09-16T08:41:39.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.251+0000 7f09b0810640 1 -- 172.21.15.160:0/591365433 >> 172.21.15.160:0/591365433 conn(0x7f09ac0fcfb0 msgr2=0x7f09ac0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:41:39.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.251+0000 7f09b0810640 1 -- 172.21.15.160:0/591365433 shutdown_connections 2024-09-16T08:41:39.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.251+0000 7f09b0810640 1 -- 172.21.15.160:0/591365433 wait complete. 2024-09-16T08:41:39.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.252+0000 7f09b0810640 1 Processor -- start 2024-09-16T08:41:39.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.252+0000 7f09b0810640 1 -- start start 2024-09-16T08:41:39.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.252+0000 7f09b0810640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f09ac1a2cb0 con 0x7f09ac105b60 2024-09-16T08:41:39.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.252+0000 7f09b0810640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f09ac1a3eb0 con 0x7f09ac1016e0 2024-09-16T08:41:39.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.252+0000 7f09b0810640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f09ac1a50b0 con 0x7f09ac1097f0 2024-09-16T08:41:39.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.252+0000 7f09aad9d640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f09ac1097f0 0x7f09ac1a13b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:54216/0 (socket says 172.21.15.160:54216) 2024-09-16T08:41:39.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.252+0000 7f09a3fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f09ac105b60 0x7f09ac19dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:48388/0 (socket says 172.21.15.160:48388) 2024-09-16T08:41:39.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.252+0000 7f09aad9d640 1 -- 172.21.15.160:0/2106503470 learned_addr learned my addr 172.21.15.160:0/2106503470 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:41:39.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.252+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3442525045 0 0) 0x7f09ac1a50b0 con 0x7f09ac1097f0 2024-09-16T08:41:39.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.252+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0998003520 con 0x7f09ac1097f0 2024-09-16T08:41:39.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.252+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3132339482 0 0) 0x7f09ac1a3eb0 con 0x7f09ac1016e0 2024-09-16T08:41:39.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.252+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09ac1a50b0 con 0x7f09ac1016e0 2024-09-16T08:41:39.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.252+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1931197546 0 0) 0x7f09ac1a2cb0 con 0x7f09ac105b60 2024-09-16T08:41:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.252+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09ac1a3eb0 con 0x7f09ac105b60 2024-09-16T08:41:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1268365907 0 0) 0x7f0998003520 con 0x7f09ac1097f0 2024-09-16T08:41:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f09ac1a2cb0 con 0x7f09ac1097f0 2024-09-16T08:41:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f09a4001690 con 0x7f09ac1097f0 2024-09-16T08:41:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3094572486 0 0) 0x7f09ac1a50b0 con 0x7f09ac1016e0 2024-09-16T08:41:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0998003520 con 0x7f09ac1016e0 2024-09-16T08:41:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2195719055 0 0) 0x7f09ac1a3eb0 con 0x7f09ac105b60 2024-09-16T08:41:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f09ac1a50b0 con 0x7f09ac105b60 2024-09-16T08:41:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f099c0017f0 con 0x7f09ac1016e0 2024-09-16T08:41:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f09940014b0 con 0x7f09ac105b60 2024-09-16T08:41:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3446397824 0 0) 0x7f09ac1a2cb0 con 0x7f09ac1097f0 2024-09-16T08:41:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 >> v1:172.21.15.169:6789/0 conn(0x7f09ac1016e0 legacy=0x7f09ac06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 >> v1:172.21.15.160:6789/0 conn(0x7f09ac105b60 legacy=0x7f09ac19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:41:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ac1a6350 con 0x7f09ac1097f0 2024-09-16T08:41:39.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09b0810640 1 -- 172.21.15.160:0/2106503470 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f09ac1a40e0 con 0x7f09ac1097f0 2024-09-16T08:41:39.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09b0810640 1 -- 172.21.15.160:0/2106503470 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f09ac1a46c0 con 0x7f09ac1097f0 2024-09-16T08:41:39.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f09a4003de0 con 0x7f09ac1097f0 2024-09-16T08:41:39.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.253+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f09a4005e40 con 0x7f09ac1097f0 2024-09-16T08:41:39.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.254+0000 7f09b0810640 1 -- 172.21.15.160:0/2106503470 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f098c0050b0 con 0x7f09ac1097f0 2024-09-16T08:41:39.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.254+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 13) ==== 51787+0+0 (unknown 1985690445 0 0) 0x7f09a40128a0 con 0x7f09ac1097f0 2024-09-16T08:41:39.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.255+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 3038969552 0 0) 0x7f09a404f5e0 con 0x7f09ac1097f0 2024-09-16T08:41:39.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.259+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f09a4018bb0 con 0x7f09ac1097f0 2024-09-16T08:41:39.277 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 08:41:39 smithi169 systemd[1]: Starting Ceph mgr.x for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T08:41:39.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:39.450+0000 7f09b0810640 1 -- 172.21.15.160:0/2106503470 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f098c000bc0 con 0x7f0998040a90 2024-09-16T08:41:39.533 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:39 smithi169 ceph-mon[33465]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:39.533 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 08:41:39 smithi169 podman[35111]: 2024-09-16 08:41:39.286094676 +0000 UTC m=+0.109029569 container create 3c5a1d4782bb2e7da7bc10be489ac52129ab757f0004192d3778925ff2c98c1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True) 2024-09-16T08:41:39.533 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 08:41:39 smithi169 podman[35111]: 2024-09-16 08:41:39.192951456 +0000 UTC m=+0.015886353 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T08:41:39.534 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 08:41:39 smithi169 podman[35111]: 2024-09-16 08:41:39.505084595 +0000 UTC m=+0.328019487 container init 3c5a1d4782bb2e7da7bc10be489ac52129ab757f0004192d3778925ff2c98c1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T08:41:39.534 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 08:41:39 smithi169 podman[35111]: 2024-09-16 08:41:39.510144023 +0000 UTC m=+0.333078916 container start 3c5a1d4782bb2e7da7bc10be489ac52129ab757f0004192d3778925ff2c98c1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-16T08:41:39.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:39 smithi160 ceph-mon[30137]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:39.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:39 smithi160 ceph-mon[40028]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:39.849 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 08:41:39 smithi169 ceph-mgr[35149]: -- 172.21.15.169:0/3750891547 <== mon.1 v1:172.21.15.169:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (unknown 877819864 0 0) 0x560a31e5ed20 con 0x560a31e3c800 2024-09-16T08:41:39.849 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 08:41:39 smithi169 bash[35111]: 3c5a1d4782bb2e7da7bc10be489ac52129ab757f0004192d3778925ff2c98c1a 2024-09-16T08:41:39.849 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 08:41:39 smithi169 systemd[1]: Started Ceph mgr.x for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:41:40.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[30137]: from='client.34109 v1:172.21.15.160:0/2106503470' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:41:40.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:41:40.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:41:40.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:40.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[40028]: from='client.34109 v1:172.21.15.160:0/2106503470' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:41:40.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:41:40.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:41:40.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:40.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:40.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:40.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:40.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:40.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:40 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:40.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:40 smithi169 ceph-mon[33465]: from='client.34109 v1:172.21.15.160:0/2106503470' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:41:40.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:40 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:41:40.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:40 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:41:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:40 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:40 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:40 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:40 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:40 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:40 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:42.035 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[30137]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:42.035 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:42.035 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:42.035 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:42.035 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:41:42.035 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:42.035 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:42.036 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:42.036 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:41:42.036 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:42.036 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:42.036 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[40028]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:42.036 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:42.036 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:42.036 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:42.036 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:41:42.036 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:42.036 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:42.037 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:42.037 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:41:42.037 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:42.037 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:42.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:41 smithi169 ceph-mon[33465]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:42.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:42.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:42.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:42.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:41:42.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:42.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:42.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:42.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:41:42.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:42.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:43.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/502296139' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6e77e8c1-25c0-4842-90a7-48769d62dbcf"} : dispatch 2024-09-16T08:41:43.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/502296139' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e77e8c1-25c0-4842-90a7-48769d62dbcf"}]': finished 2024-09-16T08:41:43.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:42 smithi169 ceph-mon[33465]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T08:41:43.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:42 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:41:43.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3658987624' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:41:43.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/502296139' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6e77e8c1-25c0-4842-90a7-48769d62dbcf"} : dispatch 2024-09-16T08:41:43.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/502296139' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e77e8c1-25c0-4842-90a7-48769d62dbcf"}]': finished 2024-09-16T08:41:43.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:42 smithi160 ceph-mon[40028]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T08:41:43.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:42 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:41:43.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3658987624' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:41:43.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/502296139' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6e77e8c1-25c0-4842-90a7-48769d62dbcf"} : dispatch 2024-09-16T08:41:43.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/502296139' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e77e8c1-25c0-4842-90a7-48769d62dbcf"}]': finished 2024-09-16T08:41:43.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:42 smithi160 ceph-mon[30137]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T08:41:43.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:42 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:41:43.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3658987624' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:41:44.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:43 smithi169 ceph-mon[33465]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:44.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:43 smithi160 ceph-mon[30137]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:44.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:43 smithi160 ceph-mon[40028]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:44.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:41:44.719+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mon.2 v1:172.21.15.160:6790/0 10 ==== mgrmap(e 14) ==== 102989+0+0 (unknown 1310866689 0 0) 0x7f09a404dee0 con 0x7f09ac1097f0 2024-09-16T08:41:45.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:44 smithi169 ceph-mon[33465]: Standby manager daemon x started 2024-09-16T08:41:45.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:44 smithi169 ceph-mon[33465]: from='mgr.? v1:172.21.15.169:0/3432703278' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-16T08:41:45.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:44 smithi169 ceph-mon[33465]: from='mgr.? v1:172.21.15.169:0/3432703278' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-16T08:41:45.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:44 smithi169 ceph-mon[33465]: from='mgr.? v1:172.21.15.169:0/3432703278' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-16T08:41:45.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:44 smithi169 ceph-mon[33465]: from='mgr.? v1:172.21.15.169:0/3432703278' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-16T08:41:45.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:44 smithi160 ceph-mon[30137]: Standby manager daemon x started 2024-09-16T08:41:45.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:44 smithi160 ceph-mon[30137]: from='mgr.? v1:172.21.15.169:0/3432703278' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-16T08:41:45.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:44 smithi160 ceph-mon[30137]: from='mgr.? v1:172.21.15.169:0/3432703278' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-16T08:41:45.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:44 smithi160 ceph-mon[30137]: from='mgr.? v1:172.21.15.169:0/3432703278' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-16T08:41:45.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:44 smithi160 ceph-mon[30137]: from='mgr.? v1:172.21.15.169:0/3432703278' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-16T08:41:45.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:44 smithi160 ceph-mon[40028]: Standby manager daemon x started 2024-09-16T08:41:45.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:44 smithi160 ceph-mon[40028]: from='mgr.? v1:172.21.15.169:0/3432703278' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-16T08:41:45.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:44 smithi160 ceph-mon[40028]: from='mgr.? v1:172.21.15.169:0/3432703278' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-16T08:41:45.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:44 smithi160 ceph-mon[40028]: from='mgr.? v1:172.21.15.169:0/3432703278' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-16T08:41:45.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:44 smithi160 ceph-mon[40028]: from='mgr.? v1:172.21.15.169:0/3432703278' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-16T08:41:45.917 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:45 smithi160 ceph-mon[30137]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:45.917 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:45 smithi160 ceph-mon[30137]: mgrmap e14: y(active, since 58s), standbys: x 2024-09-16T08:41:45.917 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:45 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-16T08:41:45.917 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:45 smithi160 ceph-mon[40028]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:45.918 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:45 smithi160 ceph-mon[40028]: mgrmap e14: y(active, since 58s), standbys: x 2024-09-16T08:41:45.918 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:45 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-16T08:41:46.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:45 smithi169 ceph-mon[33465]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:46.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:45 smithi169 ceph-mon[33465]: mgrmap e14: y(active, since 58s), standbys: x 2024-09-16T08:41:46.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:45 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-16T08:41:47.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:47 smithi160 ceph-mon[30137]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:47.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:47 smithi160 ceph-mon[40028]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:48.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:47 smithi169 ceph-mon[33465]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:49.778 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:49 smithi160 ceph-mon[30137]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:49.778 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:49 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-16T08:41:49.779 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:49 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:49.779 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:49 smithi160 ceph-mon[40028]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:49.779 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:49 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-16T08:41:49.779 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:49 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:50.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:49 smithi169 ceph-mon[33465]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:50.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:49 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-16T08:41:50.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:49 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:41:50.842 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:50 smithi160 ceph-mon[30137]: Deploying daemon osd.0 on smithi160 2024-09-16T08:41:50.842 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:50 smithi160 ceph-mon[40028]: Deploying daemon osd.0 on smithi160 2024-09-16T08:41:51.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:50 smithi169 ceph-mon[33465]: Deploying daemon osd.0 on smithi160 2024-09-16T08:41:52.012 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:51 smithi160 ceph-mon[30137]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:52.013 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:51 smithi160 ceph-mon[40028]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:52.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:51 smithi169 ceph-mon[33465]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:54.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:53 smithi169 ceph-mon[33465]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:54.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:53 smithi160 ceph-mon[30137]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:54.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:53 smithi160 ceph-mon[40028]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:56.021 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:55 smithi160 ceph-mon[30137]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:56.022 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:55 smithi160 ceph-mon[40028]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:56.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:55 smithi169 ceph-mon[33465]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:56.744 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:56.744 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:56.745 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:57.033 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:57.033 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:57.033 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:57.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:41:57.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:57.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:41:57.997 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:57 smithi160 ceph-mon[30137]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:57.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:57 smithi160 ceph-mon[40028]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:41:58.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:57 smithi169 ceph-mon[33465]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:42:00.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:59 smithi169 ceph-mon[33465]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:42:00.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:59 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:00.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:59 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:00.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:59 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:00.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:59 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:42:00.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:41:59 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:00.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:59 smithi160 ceph-mon[30137]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:42:00.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:59 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:00.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:59 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:00.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:59 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:00.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:59 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:42:00.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:41:59 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:00.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:59 smithi160 ceph-mon[40028]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:42:00.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:59 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:00.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:59 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:00.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:59 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:00.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:59 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:42:00.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:41:59 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:01.848 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:01 smithi160 ceph-mon[30137]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:42:01.848 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:01 smithi160 ceph-mon[30137]: from='osd.0 v1:172.21.15.160:6801/520428698' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-16T08:42:01.849 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:01 smithi160 ceph-mon[40028]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:42:01.849 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:01 smithi160 ceph-mon[40028]: from='osd.0 v1:172.21.15.160:6801/520428698' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-16T08:42:02.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:01 smithi169 ceph-mon[33465]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:42:02.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:01 smithi169 ceph-mon[33465]: from='osd.0 v1:172.21.15.160:6801/520428698' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-16T08:42:02.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:02.160+0000 7f09a37fe640 1 -- 172.21.15.160:0/2106503470 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 3771046575) 0x7f098c000bc0 con 0x7f0998040a90 2024-09-16T08:42:02.162 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 0 on host 'smithi160' 2024-09-16T08:42:02.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:02.162+0000 7f09b0810640 1 -- 172.21.15.160:0/2106503470 >> v1:172.21.15.160:6800/1286008384 conn(0x7f0998040a90 legacy=0x7f0998042f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:42:02.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:02.162+0000 7f09b0810640 1 -- 172.21.15.160:0/2106503470 >> v1:172.21.15.160:6790/0 conn(0x7f09ac1097f0 legacy=0x7f09ac1a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:42:02.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:02.162+0000 7f09b0810640 1 -- 172.21.15.160:0/2106503470 shutdown_connections 2024-09-16T08:42:02.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:02.162+0000 7f09b0810640 1 -- 172.21.15.160:0/2106503470 >> 172.21.15.160:0/2106503470 conn(0x7f09ac0fcfb0 msgr2=0x7f09ac10bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:42:02.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:02.162+0000 7f09b0810640 1 -- 172.21.15.160:0/2106503470 shutdown_connections 2024-09-16T08:42:02.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:02.162+0000 7f09b0810640 1 -- 172.21.15.160:0/2106503470 wait complete. 2024-09-16T08:42:02.804 DEBUG:teuthology.orchestra.run.smithi160:osd.0> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.0.service 2024-09-16T08:42:02.807 INFO:tasks.cephadm:Deploying osd.1 on smithi160 with /dev/vg_nvme/lv_3... 2024-09-16T08:42:02.807 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T08:42:03.059 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[30137]: from='osd.0 v1:172.21.15.160:6801/520428698' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T08:42:03.060 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[30137]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T08:42:03.060 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[30137]: from='osd.0 v1:172.21.15.160:6801/520428698' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:42:03.060 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:42:03.060 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[30137]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:42:03.060 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:42:03.060 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:03.060 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:03.060 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[30137]: from='osd.0 v1:172.21.15.160:6801/520428698' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-09-16T08:42:03.060 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[30137]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T08:42:03.060 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:42:03.060 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:42:03.061 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[40028]: from='osd.0 v1:172.21.15.160:6801/520428698' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T08:42:03.061 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[40028]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T08:42:03.061 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[40028]: from='osd.0 v1:172.21.15.160:6801/520428698' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:42:03.061 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:42:03.061 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[40028]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:42:03.061 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:42:03.061 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:03.061 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:03.061 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[40028]: from='osd.0 v1:172.21.15.160:6801/520428698' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-09-16T08:42:03.062 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[40028]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T08:42:03.062 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:42:03.062 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:42:03.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:02 smithi169 ceph-mon[33465]: from='osd.0 v1:172.21.15.160:6801/520428698' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T08:42:03.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:02 smithi169 ceph-mon[33465]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T08:42:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:02 smithi169 ceph-mon[33465]: from='osd.0 v1:172.21.15.160:6801/520428698' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:42:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:42:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:02 smithi169 ceph-mon[33465]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:42:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:42:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:02 smithi169 ceph-mon[33465]: from='osd.0 v1:172.21.15.160:6801/520428698' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-09-16T08:42:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:02 smithi169 ceph-mon[33465]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T08:42:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:42:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:42:03.447 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 08:42:03 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-0[43659]: 2024-09-16T08:42:03.175+0000 7fa6a701c640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T08:42:03.447 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 08:42:03 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-0[43659]: 2024-09-16T08:42:03.178+0000 7fa69ee3b640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:42:03.453 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:42:04.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:04 smithi160 ceph-mon[30137]: from='osd.0 v1:172.21.15.160:6801/520428698' entity='osd.0' 2024-09-16T08:42:04.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:04 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:42:04.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:04 smithi160 ceph-mon[40028]: from='osd.0 v1:172.21.15.160:6801/520428698' entity='osd.0' 2024-09-16T08:42:04.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:04 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:42:04.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:04 smithi169 ceph-mon[33465]: from='osd.0 v1:172.21.15.160:6801/520428698' entity='osd.0' 2024-09-16T08:42:04.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:04 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:42:05.384 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:05 smithi160 ceph-mon[30137]: purged_snaps scrub starts 2024-09-16T08:42:05.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:05 smithi160 ceph-mon[30137]: purged_snaps scrub ok 2024-09-16T08:42:05.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:05 smithi160 ceph-mon[30137]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:42:05.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:05 smithi160 ceph-mon[30137]: osd.0 v1:172.21.15.160:6801/520428698 boot 2024-09-16T08:42:05.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:05 smithi160 ceph-mon[30137]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T08:42:05.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:05 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:42:05.385 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:05 smithi160 ceph-mon[40028]: purged_snaps scrub starts 2024-09-16T08:42:05.385 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:05 smithi160 ceph-mon[40028]: purged_snaps scrub ok 2024-09-16T08:42:05.385 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:05 smithi160 ceph-mon[40028]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:42:05.385 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:05 smithi160 ceph-mon[40028]: osd.0 v1:172.21.15.160:6801/520428698 boot 2024-09-16T08:42:05.385 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:05 smithi160 ceph-mon[40028]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T08:42:05.385 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:05 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:42:05.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:05 smithi169 ceph-mon[33465]: purged_snaps scrub starts 2024-09-16T08:42:05.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:05 smithi169 ceph-mon[33465]: purged_snaps scrub ok 2024-09-16T08:42:05.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:05 smithi169 ceph-mon[33465]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:42:05.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:05 smithi169 ceph-mon[33465]: osd.0 v1:172.21.15.160:6801/520428698 boot 2024-09-16T08:42:05.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:05 smithi169 ceph-mon[33465]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T08:42:05.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:05 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:42:06.191 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:42:06.220 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_3 2024-09-16T08:42:06.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:06 smithi160 ceph-mon[30137]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T08:42:06.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:06 smithi160 ceph-mon[40028]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T08:42:06.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:06 smithi169 ceph-mon[33465]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T08:42:06.791 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:42:07.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.288+0000 7f50c465f640 1 -- 172.21.15.160:0/545787974 >> v1:172.21.15.160:6790/0 conn(0x7f50bc106c90 legacy=0x7f50bc109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:42:07.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.288+0000 7f50c465f640 1 -- 172.21.15.160:0/545787974 shutdown_connections 2024-09-16T08:42:07.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.288+0000 7f50c465f640 1 -- 172.21.15.160:0/545787974 >> 172.21.15.160:0/545787974 conn(0x7f50bc0fe0d0 msgr2=0x7f50bc1004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:42:07.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.288+0000 7f50c465f640 1 -- 172.21.15.160:0/545787974 shutdown_connections 2024-09-16T08:42:07.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.288+0000 7f50c465f640 1 -- 172.21.15.160:0/545787974 wait complete. 2024-09-16T08:42:07.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.288+0000 7f50c465f640 1 Processor -- start 2024-09-16T08:42:07.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.289+0000 7f50c465f640 1 -- start start 2024-09-16T08:42:07.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.289+0000 7f50c465f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f50bc1a2cf0 con 0x7f50bc10a9f0 2024-09-16T08:42:07.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.289+0000 7f50c465f640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f50bc1a3ed0 con 0x7f50bc103ce0 2024-09-16T08:42:07.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.289+0000 7f50c465f640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f50bc1a50b0 con 0x7f50bc106c90 2024-09-16T08:42:07.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.289+0000 7f50c1bfa640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f50bc106c90 0x7f50bc19dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:44174/0 (socket says 172.21.15.160:44174) 2024-09-16T08:42:07.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.289+0000 7f50c23fb640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f50bc103ce0 0x7f50bc197c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.160:39046/0 (socket says 172.21.15.160:39046) 2024-09-16T08:42:07.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.289+0000 7f50c1bfa640 1 -- 172.21.15.160:0/2933398045 learned_addr learned my addr 172.21.15.160:0/2933398045 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:42:07.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.289+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2235950960 0 0) 0x7f50bc1a50b0 con 0x7f50bc106c90 2024-09-16T08:42:07.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.289+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50a8003520 con 0x7f50bc106c90 2024-09-16T08:42:07.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.289+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3363269639 0 0) 0x7f50bc1a3ed0 con 0x7f50bc103ce0 2024-09-16T08:42:07.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.289+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50bc1a50b0 con 0x7f50bc103ce0 2024-09-16T08:42:07.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.289+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3069451903 0 0) 0x7f50bc1a2cf0 con 0x7f50bc10a9f0 2024-09-16T08:42:07.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.289+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50bc1a3ed0 con 0x7f50bc10a9f0 2024-09-16T08:42:07.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1069648567 0 0) 0x7f50bc1a3ed0 con 0x7f50bc10a9f0 2024-09-16T08:42:07.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f50bc1a2cf0 con 0x7f50bc10a9f0 2024-09-16T08:42:07.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f50b4001b70 con 0x7f50bc10a9f0 2024-09-16T08:42:07.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1851017713 0 0) 0x7f50bc1a50b0 con 0x7f50bc103ce0 2024-09-16T08:42:07.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f50bc1a3ed0 con 0x7f50bc103ce0 2024-09-16T08:42:07.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1500792861 0 0) 0x7f50a8003520 con 0x7f50bc106c90 2024-09-16T08:42:07.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f50bc1a50b0 con 0x7f50bc106c90 2024-09-16T08:42:07.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f50b8001210 con 0x7f50bc103ce0 2024-09-16T08:42:07.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f50ac001360 con 0x7f50bc106c90 2024-09-16T08:42:07.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3996239088 0 0) 0x7f50bc1a2cf0 con 0x7f50bc10a9f0 2024-09-16T08:42:07.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 >> v1:172.21.15.160:6790/0 conn(0x7f50bc106c90 legacy=0x7f50bc19dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:42:07.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 >> v1:172.21.15.169:6789/0 conn(0x7f50bc103ce0 legacy=0x7f50bc197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:42:07.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc1a6290 con 0x7f50bc10a9f0 2024-09-16T08:42:07.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50c465f640 1 -- 172.21.15.160:0/2933398045 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f50bc1a4100 con 0x7f50bc10a9f0 2024-09-16T08:42:07.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50c465f640 1 -- 172.21.15.160:0/2933398045 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f50bc1a46e0 con 0x7f50bc10a9f0 2024-09-16T08:42:07.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f50b4002000 con 0x7f50bc10a9f0 2024-09-16T08:42:07.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.290+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f50b4004b60 con 0x7f50bc10a9f0 2024-09-16T08:42:07.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.291+0000 7f50c465f640 1 -- 172.21.15.160:0/2933398045 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a00050b0 con 0x7f50bc10a9f0 2024-09-16T08:42:07.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.292+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 14) ==== 102989+0+0 (unknown 1310866689 0 0) 0x7f50b4003970 con 0x7f50bc10a9f0 2024-09-16T08:42:07.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.292+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(9..9 src has 1..9) ==== 1541+0+0 (unknown 361049494 0 0) 0x7f50b4096d20 con 0x7f50bc10a9f0 2024-09-16T08:42:07.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.297+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f50b405f2a0 con 0x7f50bc10a9f0 2024-09-16T08:42:07.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:07.493+0000 7f50c465f640 1 -- 172.21.15.160:0/2933398045 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f50a0000bc0 con 0x7f50a807bcd0 2024-09-16T08:42:07.495 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:07 smithi160 ceph-mon[40028]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:07.496 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:07 smithi160 ceph-mon[30137]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:07.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:07 smithi169 ceph-mon[33465]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:08.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[30137]: from='client.14250 v1:172.21.15.160:0/2933398045' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:42:08.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:42:08.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:42:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:42:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:42:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:08.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[40028]: from='client.14250 v1:172.21.15.160:0/2933398045' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:42:08.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:42:08.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:42:08.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:08.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:08.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:08.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:42:08.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:08.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:08.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:42:08.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:08 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:08.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:08 smithi169 ceph-mon[33465]: from='client.14250 v1:172.21.15.160:0/2933398045' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:42:08.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:08 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:42:08.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:08 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:42:08.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:08 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:08 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:08 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:08 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:42:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:08 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:08 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:08 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:42:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:08 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:09.388 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:09 smithi160 ceph-mon[40028]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:09.388 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:09 smithi160 ceph-mon[40028]: Detected new or changed devices on smithi160 2024-09-16T08:42:09.389 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:09 smithi160 ceph-mon[40028]: Adjusting osd_memory_target on smithi160 to 16064M 2024-09-16T08:42:09.389 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:09 smithi160 ceph-mon[30137]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:09.389 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:09 smithi160 ceph-mon[30137]: Detected new or changed devices on smithi160 2024-09-16T08:42:09.389 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:09 smithi160 ceph-mon[30137]: Adjusting osd_memory_target on smithi160 to 16064M 2024-09-16T08:42:09.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:09 smithi169 ceph-mon[33465]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:09.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:09 smithi169 ceph-mon[33465]: Detected new or changed devices on smithi160 2024-09-16T08:42:09.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:09 smithi169 ceph-mon[33465]: Adjusting osd_memory_target on smithi160 to 16064M 2024-09-16T08:42:10.219 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/45548474' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "08f4dc4d-2873-45f3-a1d9-7c12f6e2c995"} : dispatch 2024-09-16T08:42:10.474 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/45548474' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08f4dc4d-2873-45f3-a1d9-7c12f6e2c995"}]': finished 2024-09-16T08:42:10.474 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:10 smithi160 ceph-mon[30137]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T08:42:10.474 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:10 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:10.475 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/45548474' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "08f4dc4d-2873-45f3-a1d9-7c12f6e2c995"} : dispatch 2024-09-16T08:42:10.475 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/45548474' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08f4dc4d-2873-45f3-a1d9-7c12f6e2c995"}]': finished 2024-09-16T08:42:10.475 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:10 smithi160 ceph-mon[40028]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T08:42:10.475 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:10 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:10.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/45548474' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "08f4dc4d-2873-45f3-a1d9-7c12f6e2c995"} : dispatch 2024-09-16T08:42:10.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/45548474' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08f4dc4d-2873-45f3-a1d9-7c12f6e2c995"}]': finished 2024-09-16T08:42:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:10 smithi169 ceph-mon[33465]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T08:42:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:10 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:11.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:11 smithi169 ceph-mon[33465]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:11.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3885288005' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:42:11.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:11 smithi160 ceph-mon[40028]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:11.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3885288005' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:42:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:11 smithi160 ceph-mon[30137]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3885288005' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:42:13.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:13 smithi169 ceph-mon[33465]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:13.657 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:13 smithi160 ceph-mon[30137]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:13.657 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:13 smithi160 ceph-mon[40028]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:15.338 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:15 smithi160 ceph-mon[30137]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:15.338 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:15 smithi160 ceph-mon[40028]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:15.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:15 smithi169 ceph-mon[33465]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:17.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:17 smithi169 ceph-mon[33465]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:17.633 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:17 smithi160 ceph-mon[30137]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:17.633 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:17 smithi160 ceph-mon[40028]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:18.478 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:18 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-16T08:42:18.478 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:18 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:18.478 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:18 smithi160 ceph-mon[30137]: Deploying daemon osd.1 on smithi160 2024-09-16T08:42:18.479 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:18 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-16T08:42:18.479 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:18 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:18.479 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:18 smithi160 ceph-mon[40028]: Deploying daemon osd.1 on smithi160 2024-09-16T08:42:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:18 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-16T08:42:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:18 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:18 smithi169 ceph-mon[33465]: Deploying daemon osd.1 on smithi160 2024-09-16T08:42:19.351 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:19 smithi160 ceph-mon[30137]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:19.351 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:19 smithi160 ceph-mon[40028]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:19.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:19 smithi169 ceph-mon[33465]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:21.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:21 smithi169 ceph-mon[33465]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:21.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:21 smithi160 ceph-mon[30137]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:21 smithi160 ceph-mon[40028]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:23.375 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:23 smithi160 ceph-mon[30137]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:23.375 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:23 smithi160 ceph-mon[40028]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:23.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:23 smithi169 ceph-mon[33465]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:25.291 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:25 smithi160 ceph-mon[30137]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:25.292 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:25 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:42:25.292 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:25 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:25.292 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:25 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:25.292 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:25 smithi160 ceph-mon[40028]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:25.292 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:25 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:42:25.292 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:25 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:25.292 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:25 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:25.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:25 smithi169 ceph-mon[33465]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:25.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:25 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:42:25.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:25 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:25.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:25 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:27.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:27 smithi169 ceph-mon[33465]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:27.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:27 smithi160 ceph-mon[30137]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:27.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:27 smithi160 ceph-mon[40028]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:28.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:28 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:28.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:28 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:28.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:28 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:28.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:28 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:42:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:28 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:28 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:28 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:28 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:28 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:42:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:28 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:28.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:28 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:28.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:28 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:28.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:28 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:28.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:28 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:42:28.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:28 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:29.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:29.484+0000 7f50b37fe640 1 -- 172.21.15.160:0/2933398045 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 3167777819) 0x7f50a0000bc0 con 0x7f50a807bcd0 2024-09-16T08:42:29.484 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 1 on host 'smithi160' 2024-09-16T08:42:29.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:29.486+0000 7f50c465f640 1 -- 172.21.15.160:0/2933398045 >> v1:172.21.15.160:6800/1286008384 conn(0x7f50a807bcd0 legacy=0x7f50a807e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:42:29.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:29.486+0000 7f50c465f640 1 -- 172.21.15.160:0/2933398045 >> v1:172.21.15.160:6789/0 conn(0x7f50bc10a9f0 legacy=0x7f50bc1a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:42:29.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:29.486+0000 7f50c465f640 1 -- 172.21.15.160:0/2933398045 shutdown_connections 2024-09-16T08:42:29.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:29.486+0000 7f50c465f640 1 -- 172.21.15.160:0/2933398045 >> 172.21.15.160:0/2933398045 conn(0x7f50bc0fe0d0 msgr2=0x7f50bc10d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:42:29.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:29.486+0000 7f50c465f640 1 -- 172.21.15.160:0/2933398045 shutdown_connections 2024-09-16T08:42:29.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:29.486+0000 7f50c465f640 1 -- 172.21.15.160:0/2933398045 wait complete. 2024-09-16T08:42:29.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:29 smithi160 ceph-mon[30137]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:29.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:29 smithi160 ceph-mon[40028]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:29.750 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:29 smithi169 ceph-mon[33465]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:30.071 DEBUG:teuthology.orchestra.run.smithi160:osd.1> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.1.service 2024-09-16T08:42:30.074 INFO:tasks.cephadm:Deploying osd.2 on smithi160 with /dev/vg_nvme/lv_2... 2024-09-16T08:42:30.074 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T08:42:30.594 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:42:30.594 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:30.594 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:30.595 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:30 smithi160 ceph-mon[30137]: from='osd.1 v1:172.21.15.160:6805/704046158' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-16T08:42:30.595 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:30 smithi160 ceph-mon[30137]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-16T08:42:30.595 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:42:30.595 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:30.595 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:30.595 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:30 smithi160 ceph-mon[40028]: from='osd.1 v1:172.21.15.160:6805/704046158' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-16T08:42:30.595 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:30 smithi160 ceph-mon[40028]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-16T08:42:30.766 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:42:30.766 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:30.767 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:30.767 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:30 smithi169 ceph-mon[33465]: from='osd.1 v1:172.21.15.160:6805/704046158' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-16T08:42:30.767 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:30 smithi169 ceph-mon[33465]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-16T08:42:30.844 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:42:31.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:31 smithi169 ceph-mon[33465]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:31.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:31 smithi169 ceph-mon[33465]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T08:42:31.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:31 smithi169 ceph-mon[33465]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T08:42:31.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:31 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:31 smithi169 ceph-mon[33465]: from='osd.1 v1:172.21.15.160:6805/704046158' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:42:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:31 smithi169 ceph-mon[33465]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:42:31.899 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:31 smithi160 ceph-mon[30137]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:31.899 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:31 smithi160 ceph-mon[30137]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T08:42:31.899 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:31 smithi160 ceph-mon[30137]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T08:42:31.899 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:31 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:31.899 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:31 smithi160 ceph-mon[30137]: from='osd.1 v1:172.21.15.160:6805/704046158' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:42:31.899 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:31 smithi160 ceph-mon[30137]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:42:31.899 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:31 smithi160 ceph-mon[40028]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:31.900 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:31 smithi160 ceph-mon[40028]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T08:42:31.900 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:31 smithi160 ceph-mon[40028]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T08:42:31.900 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:31 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:31.900 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:31 smithi160 ceph-mon[40028]: from='osd.1 v1:172.21.15.160:6805/704046158' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:42:31.900 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:31 smithi160 ceph-mon[40028]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:42:32.198 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 08:42:31 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-1[49686]: 2024-09-16T08:42:31.926+0000 7f6709871640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T08:42:32.198 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 08:42:31 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-1[49686]: 2024-09-16T08:42:31.930+0000 7f6701690640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:42:32.849 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:32 smithi160 ceph-mon[30137]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-09-16T08:42:32.849 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:32 smithi160 ceph-mon[30137]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T08:42:32.849 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:32.849 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:32.849 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:32 smithi160 ceph-mon[30137]: from='osd.1 ' entity='osd.1' 2024-09-16T08:42:32.849 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:32 smithi160 ceph-mon[40028]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-09-16T08:42:32.849 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:32 smithi160 ceph-mon[40028]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T08:42:32.849 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:32.849 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:32.849 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:32 smithi160 ceph-mon[40028]: from='osd.1 ' entity='osd.1' 2024-09-16T08:42:32.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:32 smithi169 ceph-mon[33465]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-09-16T08:42:32.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:32 smithi169 ceph-mon[33465]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T08:42:32.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:32.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:32.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:32 smithi169 ceph-mon[33465]: from='osd.1 ' entity='osd.1' 2024-09-16T08:42:33.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:33 smithi169 ceph-mon[33465]: purged_snaps scrub starts 2024-09-16T08:42:33.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:33 smithi169 ceph-mon[33465]: purged_snaps scrub ok 2024-09-16T08:42:33.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:33 smithi169 ceph-mon[33465]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:33 smithi169 ceph-mon[33465]: osd.1 v1:172.21.15.160:6805/704046158 boot 2024-09-16T08:42:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:33 smithi169 ceph-mon[33465]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T08:42:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:42:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T08:42:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:42:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:33.877 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[30137]: purged_snaps scrub starts 2024-09-16T08:42:33.877 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[30137]: purged_snaps scrub ok 2024-09-16T08:42:33.877 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[30137]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:33.878 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:33.878 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[30137]: osd.1 v1:172.21.15.160:6805/704046158 boot 2024-09-16T08:42:33.878 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[30137]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T08:42:33.878 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:33.878 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:33.878 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:33.878 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:42:33.878 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T08:42:33.878 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:33.878 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:33.878 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:42:33.878 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:33.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[40028]: purged_snaps scrub starts 2024-09-16T08:42:33.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[40028]: purged_snaps scrub ok 2024-09-16T08:42:33.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[40028]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:42:33.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:33.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[40028]: osd.1 v1:172.21.15.160:6805/704046158 boot 2024-09-16T08:42:33.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[40028]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T08:42:33.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:42:33.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:33.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:33.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:42:33.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T08:42:33.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:33.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:33.880 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:42:33.880 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:34.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:34 smithi169 ceph-mon[33465]: Detected new or changed devices on smithi160 2024-09-16T08:42:34.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:34 smithi169 ceph-mon[33465]: Adjusting osd_memory_target on smithi160 to 8032M 2024-09-16T08:42:34.941 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:34 smithi160 ceph-mon[30137]: Detected new or changed devices on smithi160 2024-09-16T08:42:34.941 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:34 smithi160 ceph-mon[30137]: Adjusting osd_memory_target on smithi160 to 8032M 2024-09-16T08:42:34.941 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:34 smithi160 ceph-mon[40028]: Detected new or changed devices on smithi160 2024-09-16T08:42:34.942 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:34 smithi160 ceph-mon[40028]: Adjusting osd_memory_target on smithi160 to 8032M 2024-09-16T08:42:35.202 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:42:35.227 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_2 2024-09-16T08:42:35.475 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:42:35.767 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:35 smithi160 ceph-mon[30137]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:35.767 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:35 smithi160 ceph-mon[30137]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T08:42:35.767 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:35 smithi160 ceph-mon[40028]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:35.768 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:35 smithi160 ceph-mon[40028]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T08:42:35.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:35 smithi169 ceph-mon[33465]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:35.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:35 smithi169 ceph-mon[33465]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T08:42:35.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.995+0000 7f4bcf58f640 1 -- 172.21.15.160:0/3054752942 >> v1:172.21.15.160:6789/0 conn(0x7f4bc8105b60 legacy=0x7f4bc8107f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:42:35.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.995+0000 7f4bcf58f640 1 -- 172.21.15.160:0/3054752942 shutdown_connections 2024-09-16T08:42:35.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.995+0000 7f4bcf58f640 1 -- 172.21.15.160:0/3054752942 >> 172.21.15.160:0/3054752942 conn(0x7f4bc80fcfb0 msgr2=0x7f4bc80ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:42:35.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.995+0000 7f4bcf58f640 1 -- 172.21.15.160:0/3054752942 shutdown_connections 2024-09-16T08:42:35.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.995+0000 7f4bcf58f640 1 -- 172.21.15.160:0/3054752942 wait complete. 2024-09-16T08:42:35.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.996+0000 7f4bcf58f640 1 Processor -- start 2024-09-16T08:42:35.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.996+0000 7f4bcf58f640 1 -- start start 2024-09-16T08:42:35.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.996+0000 7f4bcf58f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f4bc81a1d20 con 0x7f4bc81016e0 2024-09-16T08:42:35.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.996+0000 7f4bcf58f640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f4bc81a2f00 con 0x7f4bc81097f0 2024-09-16T08:42:35.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.996+0000 7f4bcf58f640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f4bc81a40e0 con 0x7f4bc8105b60 2024-09-16T08:42:35.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.996+0000 7f4bcd32b640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f4bc81016e0 0x7f4bc8196c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:60240/0 (socket says 172.21.15.160:60240) 2024-09-16T08:42:35.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.996+0000 7f4bcd32b640 1 -- 172.21.15.160:0/1405478893 learned_addr learned my addr 172.21.15.160:0/1405478893 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:42:35.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.996+0000 7f4bccb2a640 1 --1- 172.21.15.160:0/1405478893 >> v1:172.21.15.160:6790/0 conn(0x7f4bc8105b60 0x7f4bc819ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:59704/0 (socket says 172.21.15.160:59704) 2024-09-16T08:42:35.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2990846499 0 0) 0x7f4bc81a40e0 con 0x7f4bc8105b60 2024-09-16T08:42:35.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bb4005020 con 0x7f4bc8105b60 2024-09-16T08:42:35.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3906880050 0 0) 0x7f4bc81a1d20 con 0x7f4bc81016e0 2024-09-16T08:42:35.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bc81a40e0 con 0x7f4bc81016e0 2024-09-16T08:42:35.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 847481673 0 0) 0x7f4bc81a2f00 con 0x7f4bc81097f0 2024-09-16T08:42:35.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bc81a1d20 con 0x7f4bc81097f0 2024-09-16T08:42:35.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3600142060 0 0) 0x7f4bb4005020 con 0x7f4bc8105b60 2024-09-16T08:42:35.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4bc81a2f00 con 0x7f4bc8105b60 2024-09-16T08:42:35.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 124033914 0 0) 0x7f4bc81a40e0 con 0x7f4bc81016e0 2024-09-16T08:42:36.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4bb4005020 con 0x7f4bc81016e0 2024-09-16T08:42:36.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f4bbc001490 con 0x7f4bc8105b60 2024-09-16T08:42:36.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4128729370 0 0) 0x7f4bc81a1d20 con 0x7f4bc81097f0 2024-09-16T08:42:36.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4bc81a40e0 con 0x7f4bc81097f0 2024-09-16T08:42:36.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f4bc4001270 con 0x7f4bc81016e0 2024-09-16T08:42:36.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f4bc0001ab0 con 0x7f4bc81097f0 2024-09-16T08:42:36.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2518985916 0 0) 0x7f4bc81a2f00 con 0x7f4bc8105b60 2024-09-16T08:42:36.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 >> v1:172.21.15.169:6789/0 conn(0x7f4bc81097f0 legacy=0x7f4bc81a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:42:36.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 >> v1:172.21.15.160:6789/0 conn(0x7f4bc81016e0 legacy=0x7f4bc8196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:42:36.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc81a52c0 con 0x7f4bc8105b60 2024-09-16T08:42:36.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bcf58f640 1 -- 172.21.15.160:0/1405478893 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc81a1f50 con 0x7f4bc8105b60 2024-09-16T08:42:36.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bcf58f640 1 -- 172.21.15.160:0/1405478893 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f4bc81a2530 con 0x7f4bc8105b60 2024-09-16T08:42:36.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f4bbc001a80 con 0x7f4bc8105b60 2024-09-16T08:42:36.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.997+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f4bbc004b30 con 0x7f4bc8105b60 2024-09-16T08:42:36.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.998+0000 7f4bcf58f640 1 -- 172.21.15.160:0/1405478893 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bac0050b0 con 0x7f4bc8105b60 2024-09-16T08:42:36.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.999+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 14) ==== 102989+0+0 (unknown 1310866689 0 0) 0x7f4bbc004d70 con 0x7f4bc8105b60 2024-09-16T08:42:36.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:35.999+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(14..14 src has 1..14) ==== 1833+0+0 (unknown 823338584 0 0) 0x7f4bbc098140 con 0x7f4bc8105b60 2024-09-16T08:42:36.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:36.004+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f4bbc060580 con 0x7f4bc8105b60 2024-09-16T08:42:36.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:36.188+0000 7f4bcf58f640 1 -- 172.21.15.160:0/1405478893 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4bac000bc0 con 0x7f4bb407dd40 2024-09-16T08:42:36.797 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:36 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:42:36.797 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:36 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:42:36.797 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:36 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:36.798 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:36 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:42:36.798 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:36 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:42:36.798 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:36 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:36.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:36 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:42:36.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:36 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:42:36.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:36 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:37.525 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:37 smithi160 ceph-mon[30137]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:37.526 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:37 smithi160 ceph-mon[30137]: from='client.34169 v1:172.21.15.160:0/1405478893' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:42:37.526 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:37 smithi160 ceph-mon[40028]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:37.526 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:37 smithi160 ceph-mon[40028]: from='client.34169 v1:172.21.15.160:0/1405478893' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:42:37.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:37 smithi169 ceph-mon[33465]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:37.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:37 smithi169 ceph-mon[33465]: from='client.34169 v1:172.21.15.160:0/1405478893' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:42:38.824 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4122963168' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0ba6c8a1-5676-4e1d-994a-a44a9b7d1996"} : dispatch 2024-09-16T08:42:38.824 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:38 smithi160 ceph-mon[30137]: from='client.34175 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0ba6c8a1-5676-4e1d-994a-a44a9b7d1996"} : dispatch 2024-09-16T08:42:38.824 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:38 smithi160 ceph-mon[30137]: from='client.34175 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ba6c8a1-5676-4e1d-994a-a44a9b7d1996"}]': finished 2024-09-16T08:42:38.824 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4122963168' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0ba6c8a1-5676-4e1d-994a-a44a9b7d1996"} : dispatch 2024-09-16T08:42:38.824 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:38 smithi160 ceph-mon[40028]: from='client.34175 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0ba6c8a1-5676-4e1d-994a-a44a9b7d1996"} : dispatch 2024-09-16T08:42:38.824 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:38 smithi160 ceph-mon[40028]: from='client.34175 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ba6c8a1-5676-4e1d-994a-a44a9b7d1996"}]': finished 2024-09-16T08:42:38.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4122963168' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0ba6c8a1-5676-4e1d-994a-a44a9b7d1996"} : dispatch 2024-09-16T08:42:38.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:38 smithi169 ceph-mon[33465]: from='client.34175 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0ba6c8a1-5676-4e1d-994a-a44a9b7d1996"} : dispatch 2024-09-16T08:42:38.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:38 smithi169 ceph-mon[33465]: from='client.34175 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ba6c8a1-5676-4e1d-994a-a44a9b7d1996"}]': finished 2024-09-16T08:42:39.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:39 smithi169 ceph-mon[33465]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:39.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:39 smithi169 ceph-mon[33465]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T08:42:39.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:39 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:42:39.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1656246072' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:42:39.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:39 smithi160 ceph-mon[30137]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:39.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:39 smithi160 ceph-mon[30137]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T08:42:39.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:39 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:42:39.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1656246072' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:42:39.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:39 smithi160 ceph-mon[40028]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:39.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:39 smithi160 ceph-mon[40028]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T08:42:39.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:39 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:42:39.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1656246072' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:42:41.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:41 smithi160 ceph-mon[30137]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:41.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:41 smithi160 ceph-mon[40028]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:41.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:41 smithi169 ceph-mon[33465]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:43.669 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:43 smithi160 ceph-mon[30137]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:43.669 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:43 smithi160 ceph-mon[40028]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:43.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:43 smithi169 ceph-mon[33465]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:45.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:45 smithi169 ceph-mon[33465]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:45.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:45 smithi160 ceph-mon[30137]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:45.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:45 smithi160 ceph-mon[40028]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:46.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:46 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-16T08:42:46.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:46 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:46.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:46 smithi169 ceph-mon[33465]: Deploying daemon osd.2 on smithi160 2024-09-16T08:42:46.942 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:46 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-16T08:42:46.942 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:46 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:46.942 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:46 smithi160 ceph-mon[30137]: Deploying daemon osd.2 on smithi160 2024-09-16T08:42:46.943 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:46 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-16T08:42:46.943 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:46 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:46.943 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:46 smithi160 ceph-mon[40028]: Deploying daemon osd.2 on smithi160 2024-09-16T08:42:47.558 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:47 smithi160 ceph-mon[30137]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:47.558 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:47 smithi160 ceph-mon[40028]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:47.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:47 smithi169 ceph-mon[33465]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:49.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:49 smithi169 ceph-mon[33465]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:49.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:49 smithi160 ceph-mon[30137]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:49 smithi160 ceph-mon[40028]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:51.817 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:51 smithi160 ceph-mon[30137]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:51.818 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:51 smithi160 ceph-mon[40028]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:51.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:51 smithi169 ceph-mon[33465]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:53.669 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:53 smithi160 ceph-mon[30137]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:53.669 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:53 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:42:53.670 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:53 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:53.670 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:53 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:53.670 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:53 smithi160 ceph-mon[40028]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:53.670 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:53 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:42:53.670 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:53 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:53.670 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:53 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:53.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:53 smithi169 ceph-mon[33465]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:53.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:53 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:42:53.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:53 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:53.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:53 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:55.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:55 smithi169 ceph-mon[33465]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:55.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:55 smithi160 ceph-mon[30137]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:55.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:55 smithi160 ceph-mon[40028]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:57.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:57.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:57.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:57.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:42:57.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:57.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:56 smithi169 ceph-mon[33465]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:57.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:57.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:57.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:57.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:42:57.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:57.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:56 smithi160 ceph-mon[30137]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:57.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:57.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:57.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:42:57.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:42:57.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:57.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:56 smithi160 ceph-mon[40028]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:58.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:58.010+0000 7f4bba7fc640 1 -- 172.21.15.160:0/1405478893 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 1492010951) 0x7f4bac000bc0 con 0x7f4bb407dd40 2024-09-16T08:42:58.012 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 2 on host 'smithi160' 2024-09-16T08:42:58.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:58.013+0000 7f4bcf58f640 1 -- 172.21.15.160:0/1405478893 >> v1:172.21.15.160:6800/1286008384 conn(0x7f4bb407dd40 legacy=0x7f4bb4080200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:42:58.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:58.013+0000 7f4bcf58f640 1 -- 172.21.15.160:0/1405478893 >> v1:172.21.15.160:6790/0 conn(0x7f4bc8105b60 legacy=0x7f4bc819ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:42:58.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:58.013+0000 7f4bcf58f640 1 -- 172.21.15.160:0/1405478893 shutdown_connections 2024-09-16T08:42:58.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:58.014+0000 7f4bcf58f640 1 -- 172.21.15.160:0/1405478893 >> 172.21.15.160:0/1405478893 conn(0x7f4bc80fcfb0 msgr2=0x7f4bc810bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:42:58.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:58.014+0000 7f4bcf58f640 1 -- 172.21.15.160:0/1405478893 shutdown_connections 2024-09-16T08:42:58.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:42:58.014+0000 7f4bcf58f640 1 -- 172.21.15.160:0/1405478893 wait complete. 2024-09-16T08:42:58.629 DEBUG:teuthology.orchestra.run.smithi160:osd.2> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T08:42:58.632 INFO:tasks.cephadm:Deploying osd.3 on smithi160 with /dev/vg_nvme/lv_1... 2024-09-16T08:42:58.632 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T08:42:59.163 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:58 smithi160 ceph-mon[30137]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:59.163 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:58 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:42:59.163 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:58 smithi160 ceph-mon[30137]: from='osd.2 v1:172.21.15.160:6809/2138225188' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T08:42:59.163 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:58 smithi160 ceph-mon[30137]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T08:42:59.164 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:58 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:59.164 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:58 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:59.164 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:58 smithi160 ceph-mon[40028]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:59.164 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:58 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:42:59.164 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:58 smithi160 ceph-mon[40028]: from='osd.2 v1:172.21.15.160:6809/2138225188' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T08:42:59.164 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:58 smithi160 ceph-mon[40028]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T08:42:59.164 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:58 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:59.164 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:58 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:59.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:58 smithi169 ceph-mon[33465]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:42:59.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:58 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:42:59.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:58 smithi169 ceph-mon[33465]: from='osd.2 v1:172.21.15.160:6809/2138225188' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T08:42:59.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:58 smithi169 ceph-mon[33465]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T08:42:59.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:58 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:59.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:58 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:42:59.785 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:43:00.014 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:59 smithi160 ceph-mon[40028]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T08:43:00.014 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:59 smithi160 ceph-mon[40028]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T08:43:00.014 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:59 smithi160 ceph-mon[40028]: from='osd.2 v1:172.21.15.160:6809/2138225188' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:43:00.015 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:59 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:43:00.015 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:59 smithi160 ceph-mon[40028]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:43:00.015 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:42:59 smithi160 ceph-mon[40028]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-09-16T08:43:00.015 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:59 smithi160 ceph-mon[30137]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T08:43:00.015 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:59 smithi160 ceph-mon[30137]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T08:43:00.015 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:59 smithi160 ceph-mon[30137]: from='osd.2 v1:172.21.15.160:6809/2138225188' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:43:00.015 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:59 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:43:00.015 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:59 smithi160 ceph-mon[30137]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:43:00.015 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:42:59 smithi160 ceph-mon[30137]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-09-16T08:43:00.016 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 08:42:59 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-2[55873]: 2024-09-16T08:42:59.891+0000 7f1fd10ec640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T08:43:00.016 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 08:42:59 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-2[55873]: 2024-09-16T08:42:59.897+0000 7f1fccf13640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:43:00.250 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:59 smithi169 ceph-mon[33465]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T08:43:00.251 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:59 smithi169 ceph-mon[33465]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T08:43:00.251 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:59 smithi169 ceph-mon[33465]: from='osd.2 v1:172.21.15.160:6809/2138225188' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:43:00.251 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:59 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:43:00.251 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:59 smithi169 ceph-mon[33465]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:43:00.251 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:42:59 smithi169 ceph-mon[33465]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-09-16T08:43:01.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:00 smithi160 ceph-mon[30137]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:43:01.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:00 smithi160 ceph-mon[30137]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T08:43:01.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:00 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:43:01.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:00 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:43:01.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:00 smithi160 ceph-mon[40028]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:43:01.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:00 smithi160 ceph-mon[40028]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T08:43:01.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:00 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:43:01.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:00 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:43:01.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:00 smithi169 ceph-mon[33465]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:43:01.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:00 smithi169 ceph-mon[33465]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T08:43:01.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:00 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:43:01.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:00 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:43:02.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:01 smithi160 ceph-mon[30137]: purged_snaps scrub starts 2024-09-16T08:43:02.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:01 smithi160 ceph-mon[30137]: purged_snaps scrub ok 2024-09-16T08:43:02.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:01 smithi160 ceph-mon[30137]: osd.2 v1:172.21.15.160:6809/2138225188 boot 2024-09-16T08:43:02.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:01 smithi160 ceph-mon[30137]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T08:43:02.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:01 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:43:02.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:01 smithi160 ceph-mon[40028]: purged_snaps scrub starts 2024-09-16T08:43:02.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:01 smithi160 ceph-mon[40028]: purged_snaps scrub ok 2024-09-16T08:43:02.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:01 smithi160 ceph-mon[40028]: osd.2 v1:172.21.15.160:6809/2138225188 boot 2024-09-16T08:43:02.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:01 smithi160 ceph-mon[40028]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T08:43:02.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:01 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:43:02.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:01 smithi169 ceph-mon[33465]: purged_snaps scrub starts 2024-09-16T08:43:02.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:01 smithi169 ceph-mon[33465]: purged_snaps scrub ok 2024-09-16T08:43:02.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:01 smithi169 ceph-mon[33465]: osd.2 v1:172.21.15.160:6809/2138225188 boot 2024-09-16T08:43:02.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:01 smithi169 ceph-mon[33465]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T08:43:02.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:01 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:43:03.060 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[30137]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:43:03.060 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[30137]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T08:43:03.061 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[30137]: Detected new or changed devices on smithi160 2024-09-16T08:43:03.061 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:03.061 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:03.061 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:03.061 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:03.061 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:03.061 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[30137]: Adjusting osd_memory_target on smithi160 to 5354M 2024-09-16T08:43:03.061 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:03.061 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:03.061 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:43:03.061 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:03.061 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-16T08:43:03.062 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[40028]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:43:03.062 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[40028]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T08:43:03.062 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[40028]: Detected new or changed devices on smithi160 2024-09-16T08:43:03.062 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:03.062 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:03.062 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:03.062 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:03.062 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:03.062 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[40028]: Adjusting osd_memory_target on smithi160 to 5354M 2024-09-16T08:43:03.062 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:03.062 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:03.063 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:43:03.063 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:03.063 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-16T08:43:03.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:02 smithi169 ceph-mon[33465]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:43:03.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:02 smithi169 ceph-mon[33465]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T08:43:03.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:02 smithi169 ceph-mon[33465]: Detected new or changed devices on smithi160 2024-09-16T08:43:03.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:02 smithi169 ceph-mon[33465]: Adjusting osd_memory_target on smithi160 to 5354M 2024-09-16T08:43:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:43:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-16T08:43:03.604 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:43:03.629 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_1 2024-09-16T08:43:03.880 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:43:04.254 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:04 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T08:43:04.254 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:04 smithi169 ceph-mon[33465]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T08:43:04.254 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:04 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-16T08:43:04.254 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:04 smithi169 sudo[35601]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:43:04.254 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:04 smithi169 sudo[35601]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:43:04.254 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:04 smithi169 sudo[35601]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:43:04.254 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:04 smithi169 sudo[35601]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:43:04.256 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:04 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T08:43:04.256 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:04 smithi160 ceph-mon[40028]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T08:43:04.256 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:04 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-16T08:43:04.256 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:04 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T08:43:04.256 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:04 smithi160 ceph-mon[30137]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T08:43:04.256 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:04 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-16T08:43:04.256 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59664]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:43:04.257 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59664]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:43:04.257 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59664]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:43:04.257 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59664]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:43:04.257 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59667]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:43:04.257 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59667]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:43:04.257 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59667]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:43:04.257 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59667]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:43:04.257 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59625]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:43:04.257 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59625]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:43:04.257 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59625]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:43:04.258 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59625]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:43:04.258 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59628]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:43:04.258 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59628]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:43:04.258 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59628]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:43:04.258 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59628]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:43:04.258 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59645]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:43:04.258 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59645]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:43:04.258 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59645]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:43:04.258 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59645]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:43:04.258 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59655]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:43:04.259 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59655]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:43:04.259 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59655]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:43:04.259 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59655]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:43:04.259 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59658]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:43:04.259 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59658]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:43:04.259 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59658]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:43:04.259 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59658]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:43:04.259 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59661]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:43:04.259 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59661]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:43:04.259 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59661]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:43:04.259 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59661]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:43:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.350+0000 7fb7cb771640 1 -- 172.21.15.160:0/3788472131 >> v1:172.21.15.160:6790/0 conn(0x7fb7c41016e0 legacy=0x7fb7c4101ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:43:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.350+0000 7fb7cb771640 1 -- 172.21.15.160:0/3788472131 shutdown_connections 2024-09-16T08:43:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.350+0000 7fb7cb771640 1 -- 172.21.15.160:0/3788472131 >> 172.21.15.160:0/3788472131 conn(0x7fb7c40fcfb0 msgr2=0x7fb7c40ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:43:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.350+0000 7fb7cb771640 1 -- 172.21.15.160:0/3788472131 shutdown_connections 2024-09-16T08:43:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.350+0000 7fb7cb771640 1 -- 172.21.15.160:0/3788472131 wait complete. 2024-09-16T08:43:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7cb771640 1 Processor -- start 2024-09-16T08:43:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7cb771640 1 -- start start 2024-09-16T08:43:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7cb771640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb7c41a4fd0 con 0x7fb7c4105b60 2024-09-16T08:43:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7cb771640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb7c41a61d0 con 0x7fb7c41097f0 2024-09-16T08:43:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7cb771640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb7c41a73d0 con 0x7fb7c41016e0 2024-09-16T08:43:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7c950d640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7fb7c41016e0 0x7fb7c410b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:33156/0 (socket says 172.21.15.160:33156) 2024-09-16T08:43:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7c950d640 1 -- 172.21.15.160:0/1797614111 learned_addr learned my addr 172.21.15.160:0/1797614111 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:43:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7c8d0c640 1 --1- 172.21.15.160:0/1797614111 >> v1:172.21.15.160:6789/0 conn(0x7fb7c4105b60 0x7fb7c419fed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:55198/0 (socket says 172.21.15.160:55198) 2024-09-16T08:43:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2916298061 0 0) 0x7fb7c41a4fd0 con 0x7fb7c4105b60 2024-09-16T08:43:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7b0003520 con 0x7fb7c4105b60 2024-09-16T08:43:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1853078761 0 0) 0x7fb7c41a73d0 con 0x7fb7c41016e0 2024-09-16T08:43:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7c41a4fd0 con 0x7fb7c41016e0 2024-09-16T08:43:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4058895613 0 0) 0x7fb7c41a61d0 con 0x7fb7c41097f0 2024-09-16T08:43:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7c41a73d0 con 0x7fb7c41097f0 2024-09-16T08:43:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1611775364 0 0) 0x7fb7b0003520 con 0x7fb7c4105b60 2024-09-16T08:43:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb7c41a61d0 con 0x7fb7c4105b60 2024-09-16T08:43:04.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2707219856 0 0) 0x7fb7c41a4fd0 con 0x7fb7c41016e0 2024-09-16T08:43:04.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb7b0003520 con 0x7fb7c41016e0 2024-09-16T08:43:04.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.351+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb7b8001860 con 0x7fb7c4105b60 2024-09-16T08:43:04.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.352+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb7c0001af0 con 0x7fb7c41016e0 2024-09-16T08:43:04.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.352+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 957547891 0 0) 0x7fb7c41a73d0 con 0x7fb7c41097f0 2024-09-16T08:43:04.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.352+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb7c41a4fd0 con 0x7fb7c41097f0 2024-09-16T08:43:04.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.352+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb7bc001b70 con 0x7fb7c41097f0 2024-09-16T08:43:04.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.352+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 562387407 0 0) 0x7fb7c41a61d0 con 0x7fb7c4105b60 2024-09-16T08:43:04.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.352+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 >> v1:172.21.15.160:6790/0 conn(0x7fb7c41016e0 legacy=0x7fb7c410b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:43:04.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.352+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 >> v1:172.21.15.169:6789/0 conn(0x7fb7c41097f0 legacy=0x7fb7c41a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:43:04.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.352+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c41a8670 con 0x7fb7c4105b60 2024-09-16T08:43:04.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.352+0000 7fb7cb771640 1 -- 172.21.15.160:0/1797614111 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c41a7600 con 0x7fb7c4105b60 2024-09-16T08:43:04.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.352+0000 7fb7cb771640 1 -- 172.21.15.160:0/1797614111 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb7c41a7be0 con 0x7fb7c4105b60 2024-09-16T08:43:04.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.352+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fb7b8003a80 con 0x7fb7c4105b60 2024-09-16T08:43:04.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.352+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb7b8005b10 con 0x7fb7c4105b60 2024-09-16T08:43:04.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.353+0000 7fb7cb771640 1 -- 172.21.15.160:0/1797614111 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a80050b0 con 0x7fb7c4105b60 2024-09-16T08:43:04.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.353+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 14) ==== 102989+0+0 (unknown 1310866689 0 0) 0x7fb7b8003610 con 0x7fb7c4105b60 2024-09-16T08:43:04.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.354+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 2511+0+0 (unknown 1048159687 0 0) 0x7fb7b80980a0 con 0x7fb7c4105b60 2024-09-16T08:43:04.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.358+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fb7b8060200 con 0x7fb7c4105b60 2024-09-16T08:43:04.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:04.547+0000 7fb7cb771640 1 -- 172.21.15.160:0/1797614111 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb7a8000bc0 con 0x7fb7b007bb70 2024-09-16T08:43:04.548 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59677]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:43:04.549 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59677]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:43:04.549 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59677]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:43:04.549 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59677]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:43:04.549 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59680]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:43:04.549 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59680]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:43:04.549 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59680]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:43:04.549 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:04 smithi160 sudo[59680]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:43:04.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:04 smithi169 sudo[35604]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:43:04.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:04 smithi169 sudo[35604]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:43:04.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:04 smithi169 sudo[35604]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:43:04.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:04 smithi169 sudo[35604]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:43:05.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:05.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:43:05.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T08:43:05.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T08:43:05.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:05.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:43:05.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:43:05.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:43:05.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:43:05.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:43:05.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:43:05.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:43:05.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:43:05.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:43:05.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:43:05.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:05.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:43:05.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:43:05.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:43:05.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:43:05.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:43:05.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='client.14313 v1:172.21.15.160:0/1797614111' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:43:05.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:43:05.200 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:43:05.200 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:05.200 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:05.200 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:43:05.200 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T08:43:05.200 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T08:43:05.200 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:05.200 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:43:05.200 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:43:05.200 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:43:05.201 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:43:05.201 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:43:05.201 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:43:05.201 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:43:05.201 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:43:05.201 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:43:05.201 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:43:05.201 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:05.201 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:43:05.201 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:43:05.201 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:43:05.201 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:43:05.201 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:43:05.202 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='client.14313 v1:172.21.15.160:0/1797614111' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:43:05.202 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:43:05.202 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:43:05.202 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:05 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:05.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:05.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:43:05.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T08:43:05.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T08:43:05.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:05.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:43:05.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:43:05.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:43:05.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:43:05.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:43:05.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:43:05.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:43:05.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:43:05.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:43:05.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:43:05.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:05.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:43:05.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:43:05.352 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:43:05.352 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:43:05.352 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:43:05.352 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='client.14313 v1:172.21.15.160:0/1797614111' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:43:05.352 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:43:05.353 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:43:05.353 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:05 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:06.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:06.021+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mgrmap(e 15) ==== 103045+0+0 (unknown 4003825774 0 0) 0x7fb7b805bce0 con 0x7fb7c4105b60 2024-09-16T08:43:06.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:06 smithi169 ceph-mon[33465]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T08:43:06.433 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:06 smithi160 ceph-mon[30137]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T08:43:06.433 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:06 smithi160 ceph-mon[40028]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T08:43:07.348 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[40028]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:07.349 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[40028]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T08:43:07.349 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[40028]: Cluster is now healthy 2024-09-16T08:43:07.349 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[40028]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T08:43:07.349 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4073252162' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01400bbe-6dfe-4789-8b07-5bb3177eddfb"} : dispatch 2024-09-16T08:43:07.349 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[40028]: from='client.34211 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01400bbe-6dfe-4789-8b07-5bb3177eddfb"} : dispatch 2024-09-16T08:43:07.349 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[40028]: from='client.34211 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01400bbe-6dfe-4789-8b07-5bb3177eddfb"}]': finished 2024-09-16T08:43:07.349 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[40028]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T08:43:07.349 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:43:07.349 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[30137]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:07.349 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[30137]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T08:43:07.350 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[30137]: Cluster is now healthy 2024-09-16T08:43:07.350 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[30137]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T08:43:07.350 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4073252162' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01400bbe-6dfe-4789-8b07-5bb3177eddfb"} : dispatch 2024-09-16T08:43:07.350 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[30137]: from='client.34211 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01400bbe-6dfe-4789-8b07-5bb3177eddfb"} : dispatch 2024-09-16T08:43:07.350 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[30137]: from='client.34211 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01400bbe-6dfe-4789-8b07-5bb3177eddfb"}]': finished 2024-09-16T08:43:07.350 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[30137]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T08:43:07.350 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:07 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:43:07.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:07 smithi169 ceph-mon[33465]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:07.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:07 smithi169 ceph-mon[33465]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T08:43:07.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:07 smithi169 ceph-mon[33465]: Cluster is now healthy 2024-09-16T08:43:07.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:07 smithi169 ceph-mon[33465]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T08:43:07.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:07 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4073252162' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01400bbe-6dfe-4789-8b07-5bb3177eddfb"} : dispatch 2024-09-16T08:43:07.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:07 smithi169 ceph-mon[33465]: from='client.34211 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01400bbe-6dfe-4789-8b07-5bb3177eddfb"} : dispatch 2024-09-16T08:43:07.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:07 smithi169 ceph-mon[33465]: from='client.34211 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01400bbe-6dfe-4789-8b07-5bb3177eddfb"}]': finished 2024-09-16T08:43:07.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:07 smithi169 ceph-mon[33465]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T08:43:07.351 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:07 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:43:08.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3906434799' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:43:08.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3906434799' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:43:08.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3906434799' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:43:09.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:09 smithi169 ceph-mon[33465]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:09.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:09 smithi160 ceph-mon[40028]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:09.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:09 smithi160 ceph-mon[30137]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:11.052 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:11 smithi160 ceph-mon[30137]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:11.053 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:11 smithi160 ceph-mon[40028]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:11.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:11 smithi169 ceph-mon[33465]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:13.300 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:13 smithi160 ceph-mon[30137]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:13.300 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:13 smithi160 ceph-mon[40028]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:13.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:13 smithi169 ceph-mon[33465]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:14.292 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:14 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-16T08:43:14.292 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:14 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:14.292 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:14 smithi160 ceph-mon[30137]: Deploying daemon osd.3 on smithi160 2024-09-16T08:43:14.292 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:14 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-16T08:43:14.292 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:14 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:14.292 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:14 smithi160 ceph-mon[40028]: Deploying daemon osd.3 on smithi160 2024-09-16T08:43:14.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:14 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-16T08:43:14.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:14 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:14.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:14 smithi169 ceph-mon[33465]: Deploying daemon osd.3 on smithi160 2024-09-16T08:43:15.314 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:15 smithi160 ceph-mon[30137]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:15.314 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:15 smithi160 ceph-mon[40028]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:15.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:15 smithi169 ceph-mon[33465]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:17.263 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:17 smithi160 ceph-mon[30137]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:17.263 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:17 smithi160 ceph-mon[40028]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:17.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:17 smithi169 ceph-mon[33465]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:19.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:19 smithi169 ceph-mon[33465]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:19.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:19 smithi160 ceph-mon[30137]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:19.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:19 smithi160 ceph-mon[40028]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:21.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:21 smithi169 ceph-mon[33465]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:21.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:21 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:43:21.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:21 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:21.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:21 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:21.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:21 smithi160 ceph-mon[30137]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:21.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:21 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:43:21.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:21 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:21.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:21 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:21.430 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:21 smithi160 ceph-mon[40028]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:21.430 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:21 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:43:21.430 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:21 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:21.431 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:21 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:23.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:23 smithi169 ceph-mon[33465]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:23.365 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:23 smithi160 ceph-mon[30137]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:23.365 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:23 smithi160 ceph-mon[40028]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:25.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:24 smithi169 ceph-mon[33465]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:25.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:24 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:25.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:24 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:25.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:24 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:25.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:24 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:43:25.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:24 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:25.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:24 smithi160 ceph-mon[40028]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:25.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:25.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:25.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:25.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:43:25.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:25.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:24 smithi160 ceph-mon[30137]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:25.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:24 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:25.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:24 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:25.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:24 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:25.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:24 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:43:25.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:24 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:26.151 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:25 smithi160 ceph-mon[30137]: from='osd.3 v1:172.21.15.160:6813/1752204566' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-16T08:43:26.152 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:25 smithi160 ceph-mon[30137]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-16T08:43:26.152 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:25 smithi160 ceph-mon[40028]: from='osd.3 v1:172.21.15.160:6813/1752204566' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-16T08:43:26.152 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:25 smithi160 ceph-mon[40028]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-16T08:43:26.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:26.194+0000 7fb7b67fc640 1 -- 172.21.15.160:0/1797614111 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 83601779) 0x7fb7a8000bc0 con 0x7fb7b007bb70 2024-09-16T08:43:26.195 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 3 on host 'smithi160' 2024-09-16T08:43:26.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:26.196+0000 7fb7cb771640 1 -- 172.21.15.160:0/1797614111 >> v1:172.21.15.160:6800/1286008384 conn(0x7fb7b007bb70 legacy=0x7fb7b007e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:43:26.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:26.196+0000 7fb7cb771640 1 -- 172.21.15.160:0/1797614111 >> v1:172.21.15.160:6789/0 conn(0x7fb7c4105b60 legacy=0x7fb7c419fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:43:26.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:26.196+0000 7fb7cb771640 1 -- 172.21.15.160:0/1797614111 shutdown_connections 2024-09-16T08:43:26.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:26.196+0000 7fb7cb771640 1 -- 172.21.15.160:0/1797614111 >> 172.21.15.160:0/1797614111 conn(0x7fb7c40fcfb0 msgr2=0x7fb7c410cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:43:26.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:26.197+0000 7fb7cb771640 1 -- 172.21.15.160:0/1797614111 shutdown_connections 2024-09-16T08:43:26.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:43:26.197+0000 7fb7cb771640 1 -- 172.21.15.160:0/1797614111 wait complete. 2024-09-16T08:43:26.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:25 smithi169 ceph-mon[33465]: from='osd.3 v1:172.21.15.160:6813/1752204566' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-16T08:43:26.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:25 smithi169 ceph-mon[33465]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-16T08:43:26.816 DEBUG:teuthology.orchestra.run.smithi160:osd.3> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.3.service 2024-09-16T08:43:26.818 INFO:tasks.cephadm:Deploying osd.4 on smithi169 with /dev/vg_nvme/lv_4... 2024-09-16T08:43:26.818 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T08:43:27.061 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:43:27.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:27 smithi169 ceph-mon[33465]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:27.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:27 smithi169 ceph-mon[33465]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T08:43:27.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:27 smithi169 ceph-mon[33465]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T08:43:27.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:27 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:43:27.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:27 smithi169 ceph-mon[33465]: from='osd.3 v1:172.21.15.160:6813/1752204566' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:43:27.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:27 smithi169 ceph-mon[33465]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:43:27.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:27 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:43:27.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:27 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:27.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:27 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:27.127 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-3[61676]: 2024-09-16T08:43:27.006+0000 7f72cd6d4640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T08:43:27.128 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-3[61676]: 2024-09-16T08:43:27.009+0000 7f72c94fb640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:43:27.128 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[30137]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:27.128 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[30137]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T08:43:27.128 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[30137]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T08:43:27.128 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:43:27.128 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[30137]: from='osd.3 v1:172.21.15.160:6813/1752204566' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:43:27.128 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[30137]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:43:27.129 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:43:27.129 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:27.129 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:27.129 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[40028]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:27.129 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[40028]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T08:43:27.129 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[40028]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T08:43:27.129 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:43:27.130 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[40028]: from='osd.3 v1:172.21.15.160:6813/1752204566' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:43:27.130 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[40028]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi160", "root=default"]} : dispatch 2024-09-16T08:43:27.130 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:43:27.130 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:27.130 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:27 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:28.307 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:28 smithi169 ceph-mon[33465]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-09-16T08:43:28.307 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:28 smithi169 ceph-mon[33465]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T08:43:28.307 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:28 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:43:28.307 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:28 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:43:28.310 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:28 smithi160 ceph-mon[30137]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-09-16T08:43:28.310 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:28 smithi160 ceph-mon[30137]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T08:43:28.310 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:28 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:43:28.311 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:28 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:43:28.311 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:28 smithi160 ceph-mon[40028]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-09-16T08:43:28.311 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:28 smithi160 ceph-mon[40028]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T08:43:28.311 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:28 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:43:28.311 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:28 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:43:29.317 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:29 smithi160 ceph-mon[30137]: purged_snaps scrub starts 2024-09-16T08:43:29.318 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:29 smithi160 ceph-mon[30137]: purged_snaps scrub ok 2024-09-16T08:43:29.318 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:29 smithi160 ceph-mon[30137]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:29.318 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:29 smithi160 ceph-mon[30137]: osd.3 v1:172.21.15.160:6813/1752204566 boot 2024-09-16T08:43:29.318 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:29 smithi160 ceph-mon[30137]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T08:43:29.318 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:29 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:43:29.318 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:29 smithi160 ceph-mon[40028]: purged_snaps scrub starts 2024-09-16T08:43:29.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:29 smithi160 ceph-mon[40028]: purged_snaps scrub ok 2024-09-16T08:43:29.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:29 smithi160 ceph-mon[40028]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:29.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:29 smithi160 ceph-mon[40028]: osd.3 v1:172.21.15.160:6813/1752204566 boot 2024-09-16T08:43:29.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:29 smithi160 ceph-mon[40028]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T08:43:29.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:29 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:43:29.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:29 smithi169 ceph-mon[33465]: purged_snaps scrub starts 2024-09-16T08:43:29.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:29 smithi169 ceph-mon[33465]: purged_snaps scrub ok 2024-09-16T08:43:29.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:29 smithi169 ceph-mon[33465]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:43:29.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:29 smithi169 ceph-mon[33465]: osd.3 v1:172.21.15.160:6813/1752204566 boot 2024-09-16T08:43:29.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:29 smithi169 ceph-mon[33465]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T08:43:29.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:29 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:43:29.749 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:43:29.774 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_4 2024-09-16T08:43:30.016 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:43:30.348 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:30 smithi169 ceph-mon[33465]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T08:43:30.348 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:30 smithi169 ceph-mon[33465]: Detected new or changed devices on smithi160 2024-09-16T08:43:30.348 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:30.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:30.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:30.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:30.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:30.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:30.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:30 smithi169 ceph-mon[33465]: Adjusting osd_memory_target on smithi160 to 4016M 2024-09-16T08:43:30.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:30.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:30.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:43:30.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:30.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[30137]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T08:43:30.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[30137]: Detected new or changed devices on smithi160 2024-09-16T08:43:30.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:30.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:30.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:30.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:30.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:30.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:30.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[30137]: Adjusting osd_memory_target on smithi160 to 4016M 2024-09-16T08:43:30.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:30.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:30.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:43:30.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:30.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[40028]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T08:43:30.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[40028]: Detected new or changed devices on smithi160 2024-09-16T08:43:30.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:30.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:30.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:30.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:30.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:30.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:30.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[40028]: Adjusting osd_memory_target on smithi160 to 4016M 2024-09-16T08:43:30.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:30.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:30.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:43:30.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:30.456 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.432+0000 7fd57a012640 1 -- 172.21.15.169:0/1949797236 >> v1:172.21.15.169:6789/0 conn(0x7fd57410a8c0 legacy=0x7fd57410cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:43:30.457 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.432+0000 7fd57a012640 1 -- 172.21.15.169:0/1949797236 shutdown_connections 2024-09-16T08:43:30.457 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.432+0000 7fd57a012640 1 -- 172.21.15.169:0/1949797236 >> 172.21.15.169:0/1949797236 conn(0x7fd5740fe0a0 msgr2=0x7fd5741004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:43:30.457 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.432+0000 7fd57a012640 1 -- 172.21.15.169:0/1949797236 shutdown_connections 2024-09-16T08:43:30.457 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.432+0000 7fd57a012640 1 -- 172.21.15.169:0/1949797236 wait complete. 2024-09-16T08:43:30.457 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.432+0000 7fd57a012640 1 Processor -- start 2024-09-16T08:43:30.457 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd57a012640 1 -- start start 2024-09-16T08:43:30.457 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd57a012640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd5741a2f50 con 0x7fd57410a8c0 2024-09-16T08:43:30.457 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd57a012640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd5741a4130 con 0x7fd5741027b0 2024-09-16T08:43:30.457 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd57a012640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd5741a5310 con 0x7fd574106c30 2024-09-16T08:43:30.457 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd5737fe640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fd5741027b0 0x7fd574197ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:33044/0 (socket says 172.21.15.169:33044) 2024-09-16T08:43:30.457 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd5737fe640 1 -- 172.21.15.169:0/2982179212 learned_addr learned my addr 172.21.15.169:0/2982179212 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:43:30.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2591181854 0 0) 0x7fd5741a4130 con 0x7fd5741027b0 2024-09-16T08:43:30.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd55c003520 con 0x7fd5741027b0 2024-09-16T08:43:30.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1237482007 0 0) 0x7fd5741a2f50 con 0x7fd57410a8c0 2024-09-16T08:43:30.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5741a4130 con 0x7fd57410a8c0 2024-09-16T08:43:30.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4018191559 0 0) 0x7fd5741a5310 con 0x7fd574106c30 2024-09-16T08:43:30.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5741a2f50 con 0x7fd574106c30 2024-09-16T08:43:30.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3204178655 0 0) 0x7fd55c003520 con 0x7fd5741027b0 2024-09-16T08:43:30.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd5741a5310 con 0x7fd5741027b0 2024-09-16T08:43:30.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fd564001db0 con 0x7fd5741027b0 2024-09-16T08:43:30.459 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2804284579 0 0) 0x7fd5741a4130 con 0x7fd57410a8c0 2024-09-16T08:43:30.459 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd55c003520 con 0x7fd57410a8c0 2024-09-16T08:43:30.459 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fd56c004620 con 0x7fd57410a8c0 2024-09-16T08:43:30.459 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 628734425 0 0) 0x7fd5741a2f50 con 0x7fd574106c30 2024-09-16T08:43:30.459 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd5741a4130 con 0x7fd574106c30 2024-09-16T08:43:30.459 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.433+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fd568001870 con 0x7fd574106c30 2024-09-16T08:43:30.459 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.434+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 <== mon.1 v1:172.21.15.169:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2676917625 0 0) 0x7fd5741a5310 con 0x7fd5741027b0 2024-09-16T08:43:30.459 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.434+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 >> v1:172.21.15.160:6790/0 conn(0x7fd574106c30 legacy=0x7fd57419e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:43:30.459 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.434+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 >> v1:172.21.15.160:6789/0 conn(0x7fd57410a8c0 legacy=0x7fd5741a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:43:30.459 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.434+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5741a64f0 con 0x7fd5741027b0 2024-09-16T08:43:30.459 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.434+0000 7fd57a012640 1 -- 172.21.15.169:0/2982179212 --> v1:172.21.15.169:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd5741a3180 con 0x7fd5741027b0 2024-09-16T08:43:30.459 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.434+0000 7fd57a012640 1 -- 172.21.15.169:0/2982179212 --> v1:172.21.15.169:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd5741a37b0 con 0x7fd5741027b0 2024-09-16T08:43:30.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.434+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 <== mon.1 v1:172.21.15.169:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fd564002030 con 0x7fd5741027b0 2024-09-16T08:43:30.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.434+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 <== mon.1 v1:172.21.15.169:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fd564005f20 con 0x7fd5741027b0 2024-09-16T08:43:30.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.435+0000 7fd57a012640 1 -- 172.21.15.169:0/2982179212 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5600050b0 con 0x7fd5741027b0 2024-09-16T08:43:30.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.435+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 <== mon.1 v1:172.21.15.169:6789/0 7 ==== mgrmap(e 15) ==== 103045+0+0 (unknown 4003825774 0 0) 0x7fd564003c40 con 0x7fd5741027b0 2024-09-16T08:43:30.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.436+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 <== mon.1 v1:172.21.15.169:6789/0 8 ==== osd_map(27..27 src has 1..27) ==== 2803+0+0 (unknown 3892339393 0 0) 0x7fd564098870 con 0x7fd5741027b0 2024-09-16T08:43:30.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.440+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 <== mon.1 v1:172.21.15.169:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fd564060800 con 0x7fd5741027b0 2024-09-16T08:43:30.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:30.625+0000 7fd57a012640 1 -- 172.21.15.169:0/2982179212 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd560000bc0 con 0x7fd55c07bbb0 2024-09-16T08:43:31.319 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:31 smithi169 ceph-mon[33465]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:31.319 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:31 smithi169 ceph-mon[33465]: from='client.24232 v1:172.21.15.169:0/2982179212' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:43:31.319 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:31 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:43:31.319 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:31 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:43:31.319 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:31 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:31.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:31 smithi160 ceph-mon[30137]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:31 smithi160 ceph-mon[30137]: from='client.24232 v1:172.21.15.169:0/2982179212' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:43:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:31 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:43:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:31 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:43:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:31 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:31.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:31 smithi160 ceph-mon[40028]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:31.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:31 smithi160 ceph-mon[40028]: from='client.24232 v1:172.21.15.169:0/2982179212' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:43:31.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:31 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:43:31.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:31 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:43:31.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:31 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:33.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:33 smithi160 ceph-mon[40028]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:33.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.169:0/3145459507' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f0b51ca8-0851-4938-a811-667d57dbc18f"} : dispatch 2024-09-16T08:43:33.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:33 smithi160 ceph-mon[40028]: from='client.24238 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f0b51ca8-0851-4938-a811-667d57dbc18f"} : dispatch 2024-09-16T08:43:33.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:33 smithi160 ceph-mon[40028]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0b51ca8-0851-4938-a811-667d57dbc18f"}]': finished 2024-09-16T08:43:33.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:33 smithi160 ceph-mon[40028]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T08:43:33.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:33.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:33 smithi160 ceph-mon[30137]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:33.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.169:0/3145459507' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f0b51ca8-0851-4938-a811-667d57dbc18f"} : dispatch 2024-09-16T08:43:33.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:33 smithi160 ceph-mon[30137]: from='client.24238 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f0b51ca8-0851-4938-a811-667d57dbc18f"} : dispatch 2024-09-16T08:43:33.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:33 smithi160 ceph-mon[30137]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0b51ca8-0851-4938-a811-667d57dbc18f"}]': finished 2024-09-16T08:43:33.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:33 smithi160 ceph-mon[30137]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T08:43:33.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:33.282 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:33 smithi169 ceph-mon[33465]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:33.282 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.169:0/3145459507' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f0b51ca8-0851-4938-a811-667d57dbc18f"} : dispatch 2024-09-16T08:43:33.282 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:33 smithi169 ceph-mon[33465]: from='client.24238 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f0b51ca8-0851-4938-a811-667d57dbc18f"} : dispatch 2024-09-16T08:43:33.282 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:33 smithi169 ceph-mon[33465]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0b51ca8-0851-4938-a811-667d57dbc18f"}]': finished 2024-09-16T08:43:33.283 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:33 smithi169 ceph-mon[33465]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T08:43:33.283 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:34.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.169:0/4268807207' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:43:34.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.169:0/4268807207' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:43:34.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.169:0/4268807207' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:43:35.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:35 smithi169 ceph-mon[33465]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:35.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:35 smithi160 ceph-mon[30137]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:35 smithi160 ceph-mon[40028]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:37.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:37 smithi169 ceph-mon[33465]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:37.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:37 smithi160 ceph-mon[30137]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:37.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:37 smithi160 ceph-mon[40028]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:39.331 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:39 smithi169 ceph-mon[33465]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:39.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:39 smithi160 ceph-mon[30137]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:39.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:39 smithi160 ceph-mon[40028]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:41.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:41 smithi169 ceph-mon[33465]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:41.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-16T08:43:41.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:41.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:41 smithi169 ceph-mon[33465]: Deploying daemon osd.4 on smithi169 2024-09-16T08:43:41.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:41 smithi160 ceph-mon[30137]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:41.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-16T08:43:41.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:41.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:41 smithi160 ceph-mon[30137]: Deploying daemon osd.4 on smithi169 2024-09-16T08:43:41.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:41 smithi160 ceph-mon[40028]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:41.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-16T08:43:41.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:41.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:41 smithi160 ceph-mon[40028]: Deploying daemon osd.4 on smithi169 2024-09-16T08:43:43.263 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:43 smithi169 ceph-mon[33465]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:43.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:43 smithi160 ceph-mon[30137]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:43.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:43 smithi160 ceph-mon[40028]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:45.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:45 smithi169 ceph-mon[33465]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:45.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:45 smithi160 ceph-mon[30137]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:45.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:45 smithi160 ceph-mon[40028]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:47.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:47 smithi169 ceph-mon[33465]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:47.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:47 smithi160 ceph-mon[30137]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:47.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:47 smithi160 ceph-mon[40028]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:48.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:48 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:43:48.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:48 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:48.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:48 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:48.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:48 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:43:48.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:48 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:48.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:48 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:48.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:48 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:43:48.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:48 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:48.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:48 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:49.555 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:49 smithi169 ceph-mon[33465]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:49.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:49 smithi160 ceph-mon[40028]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:49.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:49 smithi160 ceph-mon[30137]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:51.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:50 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:51.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:50 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:51.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:50 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:51.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:50 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:43:51.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:50 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:51.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:50 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:51.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:50 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:51.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:50 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:51.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:50 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:43:51.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:50 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:51.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:50 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:51.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:50 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:51.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:50 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:51.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:50 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:43:51.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:50 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:51.978 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:51 smithi169 ceph-mon[33465]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:52.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:51 smithi160 ceph-mon[40028]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:52.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:51 smithi160 ceph-mon[30137]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:52.676 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:52.675+0000 7fd570ff9640 1 -- 172.21.15.169:0/2982179212 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 141490741) 0x7fd560000bc0 con 0x7fd55c07bbb0 2024-09-16T08:43:52.677 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 4 on host 'smithi169' 2024-09-16T08:43:52.678 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:52.678+0000 7fd57a012640 1 -- 172.21.15.169:0/2982179212 >> v1:172.21.15.160:6800/1286008384 conn(0x7fd55c07bbb0 legacy=0x7fd55c07e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:43:52.678 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:52.678+0000 7fd57a012640 1 -- 172.21.15.169:0/2982179212 >> v1:172.21.15.169:6789/0 conn(0x7fd5741027b0 legacy=0x7fd574197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:43:52.678 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:52.678+0000 7fd57a012640 1 -- 172.21.15.169:0/2982179212 shutdown_connections 2024-09-16T08:43:52.679 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:52.678+0000 7fd57a012640 1 -- 172.21.15.169:0/2982179212 >> 172.21.15.169:0/2982179212 conn(0x7fd5740fe0a0 msgr2=0x7fd5740fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:43:52.679 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:52.678+0000 7fd57a012640 1 -- 172.21.15.169:0/2982179212 shutdown_connections 2024-09-16T08:43:52.679 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:52.678+0000 7fd57a012640 1 -- 172.21.15.169:0/2982179212 wait complete. 2024-09-16T08:43:52.987 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:52 smithi169 ceph-mon[33465]: from='osd.4 v1:172.21.15.169:6800/1893879009' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-16T08:43:52.987 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:52 smithi169 ceph-mon[33465]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-16T08:43:52.987 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:52 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:43:52.987 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:52 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:52.987 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:52 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:53.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:52 smithi160 ceph-mon[30137]: from='osd.4 v1:172.21.15.169:6800/1893879009' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-16T08:43:53.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:52 smithi160 ceph-mon[30137]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-16T08:43:53.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:52 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:43:53.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:52 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:53.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:52 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:53.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:52 smithi160 ceph-mon[40028]: from='osd.4 v1:172.21.15.169:6800/1893879009' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-16T08:43:53.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:52 smithi160 ceph-mon[40028]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-16T08:43:53.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:52 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:43:53.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:52 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:53.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:52 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:53.285 DEBUG:teuthology.orchestra.run.smithi169:osd.4> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.4.service 2024-09-16T08:43:53.287 INFO:tasks.cephadm:Deploying osd.5 on smithi169 with /dev/vg_nvme/lv_3... 2024-09-16T08:43:53.287 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T08:43:53.802 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:53 smithi169 ceph-mon[33465]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:53.803 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:53 smithi169 ceph-mon[33465]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T08:43:53.803 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:53 smithi169 ceph-mon[33465]: from='osd.4 v1:172.21.15.169:6800/1893879009' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:43:53.803 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:53 smithi169 ceph-mon[33465]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T08:43:53.803 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:53 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:53.803 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:53 smithi169 ceph-mon[33465]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:43:54.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:53 smithi160 ceph-mon[30137]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:54.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:53 smithi160 ceph-mon[30137]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T08:43:54.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:53 smithi160 ceph-mon[30137]: from='osd.4 v1:172.21.15.169:6800/1893879009' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:43:54.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:53 smithi160 ceph-mon[30137]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T08:43:54.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:53 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:54.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:53 smithi160 ceph-mon[30137]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:43:54.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:53 smithi160 ceph-mon[40028]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:54.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:53 smithi160 ceph-mon[40028]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T08:43:54.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:53 smithi160 ceph-mon[40028]: from='osd.4 v1:172.21.15.169:6800/1893879009' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:43:54.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:53 smithi160 ceph-mon[40028]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T08:43:54.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:53 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:54.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:53 smithi160 ceph-mon[40028]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:43:54.344 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 08:43:54 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4[38096]: 2024-09-16T08:43:54.166+0000 7fee421d2640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T08:43:54.344 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 08:43:54 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4[38096]: 2024-09-16T08:43:54.169+0000 7fee39ff1640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:43:54.361 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:43:54.938 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:54 smithi169 ceph-mon[33465]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T08:43:54.938 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:54 smithi169 ceph-mon[33465]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T08:43:54.938 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:54 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:54.938 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:54 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:54.938 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:54 smithi169 ceph-mon[33465]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T08:43:54.938 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:54 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:54.938 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:54 smithi169 ceph-mon[33465]: from='osd.4 ' entity='osd.4' 2024-09-16T08:43:55.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:54 smithi160 ceph-mon[30137]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T08:43:55.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:54 smithi160 ceph-mon[30137]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T08:43:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:54 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:54 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:54 smithi160 ceph-mon[30137]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T08:43:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:54 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:54 smithi160 ceph-mon[30137]: from='osd.4 ' entity='osd.4' 2024-09-16T08:43:55.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:54 smithi160 ceph-mon[40028]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T08:43:55.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:54 smithi160 ceph-mon[40028]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T08:43:55.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:54 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:55.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:54 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:55.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:54 smithi160 ceph-mon[40028]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T08:43:55.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:54 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:55.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:54 smithi160 ceph-mon[40028]: from='osd.4 ' entity='osd.4' 2024-09-16T08:43:55.965 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:55 smithi169 ceph-mon[33465]: purged_snaps scrub starts 2024-09-16T08:43:55.966 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:55 smithi169 ceph-mon[33465]: purged_snaps scrub ok 2024-09-16T08:43:55.966 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:55 smithi169 ceph-mon[33465]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:55.966 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:55 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:55.966 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:55 smithi169 ceph-mon[33465]: osd.4 v1:172.21.15.169:6800/1893879009 boot 2024-09-16T08:43:55.966 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:55 smithi169 ceph-mon[33465]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T08:43:55.966 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:55 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:56.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:55 smithi160 ceph-mon[30137]: purged_snaps scrub starts 2024-09-16T08:43:56.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:55 smithi160 ceph-mon[30137]: purged_snaps scrub ok 2024-09-16T08:43:56.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:55 smithi160 ceph-mon[30137]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:56.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:55 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:56.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:55 smithi160 ceph-mon[30137]: osd.4 v1:172.21.15.169:6800/1893879009 boot 2024-09-16T08:43:56.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:55 smithi160 ceph-mon[30137]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T08:43:56.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:55 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:56.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:55 smithi160 ceph-mon[40028]: purged_snaps scrub starts 2024-09-16T08:43:56.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:55 smithi160 ceph-mon[40028]: purged_snaps scrub ok 2024-09-16T08:43:56.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:55 smithi160 ceph-mon[40028]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:43:56.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:55 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:56.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:55 smithi160 ceph-mon[40028]: osd.4 v1:172.21.15.169:6800/1893879009 boot 2024-09-16T08:43:56.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:55 smithi160 ceph-mon[40028]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T08:43:56.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:55 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:43:57.293 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:57 smithi169 ceph-mon[33465]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:43:57.293 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:57 smithi169 ceph-mon[33465]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T08:43:57.293 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:57 smithi169 ceph-mon[33465]: Detected new or changed devices on smithi169 2024-09-16T08:43:57.293 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:57 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:57.293 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:57 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:57.293 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:57 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:57.294 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:57 smithi169 ceph-mon[33465]: Adjusting osd_memory_target on smithi169 to 17088M 2024-09-16T08:43:57.294 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:57 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:57.294 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:57 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:57.294 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:57 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:43:57.294 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:57 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:57.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[30137]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:43:57.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[30137]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T08:43:57.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[30137]: Detected new or changed devices on smithi169 2024-09-16T08:43:57.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:57.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:57.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:57.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[30137]: Adjusting osd_memory_target on smithi169 to 17088M 2024-09-16T08:43:57.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:57.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:57.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:43:57.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:57.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[40028]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:43:57.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[40028]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T08:43:57.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[40028]: Detected new or changed devices on smithi169 2024-09-16T08:43:57.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:57.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:57.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:43:57.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[40028]: Adjusting osd_memory_target on smithi169 to 17088M 2024-09-16T08:43:57.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:57.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:43:57.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:43:57.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:57 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:43:58.210 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:43:58.234 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:58 smithi169 ceph-mon[33465]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T08:43:58.235 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_3 2024-09-16T08:43:58.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:58 smithi160 ceph-mon[40028]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T08:43:58.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:58 smithi160 ceph-mon[30137]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T08:43:58.482 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:43:58.974 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.973+0000 7fb05c964640 1 -- 172.21.15.169:0/3355133882 >> v1:172.21.15.169:6789/0 conn(0x7fb0581097b0 legacy=0x7fb05810bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:43:58.974 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.973+0000 7fb05c964640 1 -- 172.21.15.169:0/3355133882 shutdown_connections 2024-09-16T08:43:58.974 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.973+0000 7fb05c964640 1 -- 172.21.15.169:0/3355133882 >> 172.21.15.169:0/3355133882 conn(0x7fb0580fcf70 msgr2=0x7fb0580ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:43:58.974 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.973+0000 7fb05c964640 1 -- 172.21.15.169:0/3355133882 shutdown_connections 2024-09-16T08:43:58.974 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.973+0000 7fb05c964640 1 -- 172.21.15.169:0/3355133882 wait complete. 2024-09-16T08:43:58.977 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.974+0000 7fb05c964640 1 Processor -- start 2024-09-16T08:43:58.977 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.974+0000 7fb05c964640 1 -- start start 2024-09-16T08:43:58.977 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.974+0000 7fb05c964640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb0581a5030 con 0x7fb0581016a0 2024-09-16T08:43:58.977 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.974+0000 7fb05c964640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb0581a6230 con 0x7fb0581097b0 2024-09-16T08:43:58.977 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.974+0000 7fb05c964640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb0581a7430 con 0x7fb058105b20 2024-09-16T08:43:58.977 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.974+0000 7fb056d9d640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fb0581097b0 0x7fb0581a3750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:54700/0 (socket says 172.21.15.169:54700) 2024-09-16T08:43:58.977 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.974+0000 7fb056d9d640 1 -- 172.21.15.169:0/1919123050 learned_addr learned my addr 172.21.15.169:0/1919123050 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:43:58.977 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.974+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3843970951 0 0) 0x7fb0581a6230 con 0x7fb0581097b0 2024-09-16T08:43:58.977 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.974+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb040003520 con 0x7fb0581097b0 2024-09-16T08:43:58.977 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.974+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 248138557 0 0) 0x7fb0581a5030 con 0x7fb0581016a0 2024-09-16T08:43:58.978 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.974+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0581a6230 con 0x7fb0581016a0 2024-09-16T08:43:58.978 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2178733752 0 0) 0x7fb0581a7430 con 0x7fb058105b20 2024-09-16T08:43:58.978 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0581a5030 con 0x7fb058105b20 2024-09-16T08:43:58.978 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2029659616 0 0) 0x7fb040003520 con 0x7fb0581097b0 2024-09-16T08:43:58.978 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb0581a7430 con 0x7fb0581097b0 2024-09-16T08:43:58.978 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb0500014b0 con 0x7fb0581097b0 2024-09-16T08:43:58.978 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4209826600 0 0) 0x7fb0581a6230 con 0x7fb0581016a0 2024-09-16T08:43:58.978 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb040003520 con 0x7fb0581016a0 2024-09-16T08:43:58.978 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb048001c20 con 0x7fb0581016a0 2024-09-16T08:43:58.978 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3417595535 0 0) 0x7fb0581a5030 con 0x7fb058105b20 2024-09-16T08:43:58.978 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb0581a6230 con 0x7fb058105b20 2024-09-16T08:43:58.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb04c0018b0 con 0x7fb058105b20 2024-09-16T08:43:58.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 <== mon.1 v1:172.21.15.169:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1319789797 0 0) 0x7fb0581a7430 con 0x7fb0581097b0 2024-09-16T08:43:58.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 >> v1:172.21.15.160:6790/0 conn(0x7fb058105b20 legacy=0x7fb0581a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:43:58.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 >> v1:172.21.15.160:6789/0 conn(0x7fb0581016a0 legacy=0x7fb05806b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:43:58.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0581a86d0 con 0x7fb0581097b0 2024-09-16T08:43:58.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb05c964640 1 -- 172.21.15.169:0/1919123050 --> v1:172.21.15.169:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb0581a5260 con 0x7fb0581097b0 2024-09-16T08:43:58.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb05c964640 1 -- 172.21.15.169:0/1919123050 --> v1:172.21.15.169:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb0581a5890 con 0x7fb0581097b0 2024-09-16T08:43:58.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 <== mon.1 v1:172.21.15.169:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fb050001980 con 0x7fb0581097b0 2024-09-16T08:43:58.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.975+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 <== mon.1 v1:172.21.15.169:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb050004e80 con 0x7fb0581097b0 2024-09-16T08:43:58.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.976+0000 7fb05c964640 1 -- 172.21.15.169:0/1919123050 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0380050b0 con 0x7fb0581097b0 2024-09-16T08:43:58.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.977+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 <== mon.1 v1:172.21.15.169:6789/0 7 ==== mgrmap(e 15) ==== 103045+0+0 (unknown 4003825774 0 0) 0x7fb050003800 con 0x7fb0581097b0 2024-09-16T08:43:58.980 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.977+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 <== mon.1 v1:172.21.15.169:6789/0 8 ==== osd_map(34..34 src has 1..34) ==== 3205+0+0 (unknown 4149043223 0 0) 0x7fb050098690 con 0x7fb0581097b0 2024-09-16T08:43:58.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:58.981+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 <== mon.1 v1:172.21.15.169:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fb050060450 con 0x7fb0581097b0 2024-09-16T08:43:59.168 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:43:59.167+0000 7fb05c964640 1 -- 172.21.15.169:0/1919123050 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb038000bc0 con 0x7fb04007bb70 2024-09-16T08:43:59.168 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:43:59 smithi169 ceph-mon[33465]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:43:59.348 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:43:59 smithi160 ceph-mon[30137]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:43:59.349 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:43:59 smithi160 ceph-mon[40028]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:00.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:00 smithi169 ceph-mon[33465]: from='client.24259 v1:172.21.15.169:0/1919123050' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:44:00.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:00 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:44:00.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:00 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:44:00.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:00 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:00.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:00 smithi160 ceph-mon[30137]: from='client.24259 v1:172.21.15.169:0/1919123050' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:44:00.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:00 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:44:00.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:00 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:44:00.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:00 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:00.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:00 smithi160 ceph-mon[40028]: from='client.24259 v1:172.21.15.169:0/1919123050' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:44:00.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:00 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:44:00.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:00 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:44:00.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:00 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:01.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:01 smithi169 ceph-mon[33465]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-16T08:44:01.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:01 smithi160 ceph-mon[30137]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-16T08:44:01.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:01 smithi160 ceph-mon[40028]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-16T08:44:02.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:02 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.169:0/2637286880' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "15a1f6ce-7c1b-4ebd-ab8a-4bacdbe5df55"} : dispatch 2024-09-16T08:44:02.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:02 smithi160 ceph-mon[30137]: from='client.24265 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "15a1f6ce-7c1b-4ebd-ab8a-4bacdbe5df55"} : dispatch 2024-09-16T08:44:02.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:02 smithi160 ceph-mon[30137]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15a1f6ce-7c1b-4ebd-ab8a-4bacdbe5df55"}]': finished 2024-09-16T08:44:02.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:02 smithi160 ceph-mon[30137]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T08:44:02.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:02 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:44:02.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:02 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.169:0/2637286880' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "15a1f6ce-7c1b-4ebd-ab8a-4bacdbe5df55"} : dispatch 2024-09-16T08:44:02.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:02 smithi160 ceph-mon[40028]: from='client.24265 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "15a1f6ce-7c1b-4ebd-ab8a-4bacdbe5df55"} : dispatch 2024-09-16T08:44:02.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:02 smithi160 ceph-mon[40028]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15a1f6ce-7c1b-4ebd-ab8a-4bacdbe5df55"}]': finished 2024-09-16T08:44:02.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:02 smithi160 ceph-mon[40028]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T08:44:02.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:02 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:44:02.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:02 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.169:0/2637286880' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "15a1f6ce-7c1b-4ebd-ab8a-4bacdbe5df55"} : dispatch 2024-09-16T08:44:02.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:02 smithi169 ceph-mon[33465]: from='client.24265 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "15a1f6ce-7c1b-4ebd-ab8a-4bacdbe5df55"} : dispatch 2024-09-16T08:44:02.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:02 smithi169 ceph-mon[33465]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15a1f6ce-7c1b-4ebd-ab8a-4bacdbe5df55"}]': finished 2024-09-16T08:44:02.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:02 smithi169 ceph-mon[33465]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T08:44:02.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:02 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:44:03.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:03 smithi169 ceph-mon[33465]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T08:44:03.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.169:0/1225448444' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:44:03.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:03 smithi160 ceph-mon[30137]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T08:44:03.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.169:0/1225448444' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:44:03.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:03 smithi160 ceph-mon[40028]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T08:44:03.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.169:0/1225448444' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:44:05.362 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:05 smithi169 ceph-mon[33465]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T08:44:05.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:05 smithi160 ceph-mon[30137]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T08:44:05.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:05 smithi160 ceph-mon[40028]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T08:44:07.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:07 smithi169 ceph-mon[33465]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-16T08:44:07.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:07 smithi160 ceph-mon[30137]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-16T08:44:07.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:07 smithi160 ceph-mon[40028]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-16T08:44:09.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:09 smithi169 ceph-mon[33465]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:44:09.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:09 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-16T08:44:09.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:09 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:09.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:09 smithi160 ceph-mon[30137]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:44:09.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:09 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-16T08:44:09.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:09 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:09 smithi160 ceph-mon[40028]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:44:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:09 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-16T08:44:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:09 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:10.261 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:10 smithi169 ceph-mon[33465]: Deploying daemon osd.5 on smithi169 2024-09-16T08:44:10.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:10 smithi160 ceph-mon[30137]: Deploying daemon osd.5 on smithi169 2024-09-16T08:44:10.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:10 smithi160 ceph-mon[40028]: Deploying daemon osd.5 on smithi169 2024-09-16T08:44:11.435 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:11 smithi169 ceph-mon[33465]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:11.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:11 smithi160 ceph-mon[30137]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:11.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:11 smithi160 ceph-mon[40028]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:13.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:13 smithi160 ceph-mon[30137]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:13.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:13 smithi160 ceph-mon[40028]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:13 smithi169 ceph-mon[33465]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:15.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:15 smithi160 ceph-mon[30137]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:15.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:15 smithi160 ceph-mon[40028]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:15.476 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:15 smithi169 ceph-mon[33465]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:17.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:17 smithi169 ceph-mon[33465]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:17.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:17 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:44:17.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:17 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:17.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:17 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:17.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:17 smithi160 ceph-mon[30137]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:17 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:44:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:17 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:17 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:17.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:17 smithi160 ceph-mon[40028]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:17.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:17 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:44:17.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:17 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:17.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:17 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:19.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:19 smithi160 ceph-mon[30137]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:19 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:19 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:19 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:19 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:44:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:19 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:19.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:19 smithi160 ceph-mon[40028]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:19.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:19 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:19.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:19 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:19.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:19 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:19.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:19 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:44:19.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:19 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:19.575 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:19 smithi169 ceph-mon[33465]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:19.576 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:19 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:19.576 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:19 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:19.576 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:19 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:19.576 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:19 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:44:19.577 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:19 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:21.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:21 smithi160 ceph-mon[30137]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:21.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:21 smithi160 ceph-mon[40028]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:21.506 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:21 smithi169 ceph-mon[33465]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:21.551 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:21.550+0000 7fb0477fe640 1 -- 172.21.15.169:0/1919123050 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 1417352321) 0x7fb038000bc0 con 0x7fb04007bb70 2024-09-16T08:44:21.551 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 5 on host 'smithi169' 2024-09-16T08:44:21.553 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:21.553+0000 7fb05c964640 1 -- 172.21.15.169:0/1919123050 >> v1:172.21.15.160:6800/1286008384 conn(0x7fb04007bb70 legacy=0x7fb04007e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:44:21.553 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:21.553+0000 7fb05c964640 1 -- 172.21.15.169:0/1919123050 >> v1:172.21.15.169:6789/0 conn(0x7fb0581097b0 legacy=0x7fb0581a3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:44:21.553 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:21.553+0000 7fb05c964640 1 -- 172.21.15.169:0/1919123050 shutdown_connections 2024-09-16T08:44:21.554 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:21.553+0000 7fb05c964640 1 -- 172.21.15.169:0/1919123050 >> 172.21.15.169:0/1919123050 conn(0x7fb0580fcf70 msgr2=0x7fb058109c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:44:21.554 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:21.553+0000 7fb05c964640 1 -- 172.21.15.169:0/1919123050 shutdown_connections 2024-09-16T08:44:21.554 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:21.553+0000 7fb05c964640 1 -- 172.21.15.169:0/1919123050 wait complete. 2024-09-16T08:44:22.144 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:22 smithi169 ceph-mon[33465]: from='osd.5 v1:172.21.15.169:6804/4217518758' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-16T08:44:22.144 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:22 smithi169 ceph-mon[33465]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-16T08:44:22.144 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:22 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:44:22.144 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:22 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:22.144 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:22 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:22.172 DEBUG:teuthology.orchestra.run.smithi169:osd.5> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.5.service 2024-09-16T08:44:22.175 INFO:tasks.cephadm:Deploying osd.6 on smithi169 with /dev/vg_nvme/lv_2... 2024-09-16T08:44:22.175 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T08:44:22.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:22 smithi160 ceph-mon[30137]: from='osd.5 v1:172.21.15.169:6804/4217518758' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-16T08:44:22.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:22 smithi160 ceph-mon[30137]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-16T08:44:22.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:22 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:44:22.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:22 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:22.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:22 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:22.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:22 smithi160 ceph-mon[40028]: from='osd.5 v1:172.21.15.169:6804/4217518758' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-16T08:44:22.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:22 smithi160 ceph-mon[40028]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-16T08:44:22.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:22 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:44:22.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:22 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:22.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:22 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:22.971 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:44:23.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:23 smithi169 ceph-mon[33465]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:23.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:23 smithi169 ceph-mon[33465]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T08:44:23.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:23 smithi169 ceph-mon[33465]: from='osd.5 v1:172.21.15.169:6804/4217518758' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:44:23.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:23 smithi169 ceph-mon[33465]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T08:44:23.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:23 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:44:23.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:23 smithi169 ceph-mon[33465]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:44:23.350 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 08:44:23 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T08:44:23.143+0000 7f2a56b76640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T08:44:23.350 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 08:44:23 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T08:44:23.147+0000 7f2a5299d640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:44:23.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:23 smithi160 ceph-mon[30137]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:23 smithi160 ceph-mon[30137]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T08:44:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:23 smithi160 ceph-mon[30137]: from='osd.5 v1:172.21.15.169:6804/4217518758' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:44:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:23 smithi160 ceph-mon[30137]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T08:44:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:23 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:44:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:23 smithi160 ceph-mon[30137]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:44:23.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:23 smithi160 ceph-mon[40028]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:23.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:23 smithi160 ceph-mon[40028]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T08:44:23.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:23 smithi160 ceph-mon[40028]: from='osd.5 v1:172.21.15.169:6804/4217518758' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:44:23.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:23 smithi160 ceph-mon[40028]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T08:44:23.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:23 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:44:23.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:23 smithi160 ceph-mon[40028]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:44:24.424 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:24 smithi169 ceph-mon[33465]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T08:44:24.425 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:24 smithi169 ceph-mon[33465]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T08:44:24.425 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:24 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:44:24.425 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:24 smithi169 ceph-mon[33465]: osd.5 v1:172.21.15.169:6804/4217518758 boot 2024-09-16T08:44:24.425 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:24 smithi169 ceph-mon[33465]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T08:44:24.425 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:24 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:44:24.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:24 smithi160 ceph-mon[30137]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T08:44:24.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:24 smithi160 ceph-mon[30137]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T08:44:24.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:24 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:44:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:24 smithi160 ceph-mon[30137]: osd.5 v1:172.21.15.169:6804/4217518758 boot 2024-09-16T08:44:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:24 smithi160 ceph-mon[30137]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T08:44:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:24 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:44:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:24 smithi160 ceph-mon[40028]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T08:44:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:24 smithi160 ceph-mon[40028]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T08:44:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:44:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:24 smithi160 ceph-mon[40028]: osd.5 v1:172.21.15.169:6804/4217518758 boot 2024-09-16T08:44:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:24 smithi160 ceph-mon[40028]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T08:44:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:44:25.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:25 smithi160 ceph-mon[30137]: purged_snaps scrub starts 2024-09-16T08:44:25.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:25 smithi160 ceph-mon[30137]: purged_snaps scrub ok 2024-09-16T08:44:25.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:25 smithi160 ceph-mon[30137]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:25.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:25 smithi160 ceph-mon[30137]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T08:44:25.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:25 smithi160 ceph-mon[40028]: purged_snaps scrub starts 2024-09-16T08:44:25.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:25 smithi160 ceph-mon[40028]: purged_snaps scrub ok 2024-09-16T08:44:25.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:25 smithi160 ceph-mon[40028]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:25.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:25 smithi160 ceph-mon[40028]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T08:44:25.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:25 smithi169 ceph-mon[33465]: purged_snaps scrub starts 2024-09-16T08:44:25.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:25 smithi169 ceph-mon[33465]: purged_snaps scrub ok 2024-09-16T08:44:25.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:25 smithi169 ceph-mon[33465]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:44:25.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:25 smithi169 ceph-mon[33465]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T08:44:26.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[30137]: Detected new or changed devices on smithi169 2024-09-16T08:44:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:44:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T08:44:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[30137]: Adjusting osd_memory_target on smithi169 to 8544M 2024-09-16T08:44:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:44:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[30137]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T08:44:26.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[40028]: Detected new or changed devices on smithi169 2024-09-16T08:44:26.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:26.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:26.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:44:26.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T08:44:26.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[40028]: Adjusting osd_memory_target on smithi169 to 8544M 2024-09-16T08:44:26.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:26.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:26.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:44:26.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:26.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:26 smithi160 ceph-mon[40028]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T08:44:27.055 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:26 smithi169 ceph-mon[33465]: Detected new or changed devices on smithi169 2024-09-16T08:44:27.055 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:26 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:27.055 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:26 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:27.055 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:26 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:44:27.055 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:26 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T08:44:27.055 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:26 smithi169 ceph-mon[33465]: Adjusting osd_memory_target on smithi169 to 8544M 2024-09-16T08:44:27.055 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:26 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:27.055 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:26 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:27.055 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:26 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:44:27.056 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:26 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:27.056 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:26 smithi169 ceph-mon[33465]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T08:44:27.342 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:44:27.370 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_2 2024-09-16T08:44:27.618 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:44:27.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:27 smithi160 ceph-mon[30137]: pgmap v139: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:27.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:27 smithi160 ceph-mon[40028]: pgmap v139: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:28.001 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:27 smithi169 ceph-mon[33465]: pgmap v139: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:28.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.084+0000 7fac37389640 1 -- 172.21.15.169:0/3584707318 >> v1:172.21.15.169:6789/0 conn(0x7fac3010a8c0 legacy=0x7fac3010cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:44:28.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.084+0000 7fac37389640 1 -- 172.21.15.169:0/3584707318 shutdown_connections 2024-09-16T08:44:28.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.084+0000 7fac37389640 1 -- 172.21.15.169:0/3584707318 >> 172.21.15.169:0/3584707318 conn(0x7fac300fe0a0 msgr2=0x7fac301004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:44:28.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.084+0000 7fac37389640 1 -- 172.21.15.169:0/3584707318 shutdown_connections 2024-09-16T08:44:28.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.084+0000 7fac37389640 1 -- 172.21.15.169:0/3584707318 wait complete. 2024-09-16T08:44:28.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.085+0000 7fac37389640 1 Processor -- start 2024-09-16T08:44:28.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.085+0000 7fac37389640 1 -- start start 2024-09-16T08:44:28.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.085+0000 7fac37389640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fac301a2f80 con 0x7fac3010a8c0 2024-09-16T08:44:28.102 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.085+0000 7fac37389640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fac301a4160 con 0x7fac301027b0 2024-09-16T08:44:28.102 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.085+0000 7fac37389640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fac301a5340 con 0x7fac30106c30 2024-09-16T08:44:28.102 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.085+0000 7fac35125640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fac301027b0 0x7fac30197ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:55856/0 (socket says 172.21.15.169:55856) 2024-09-16T08:44:28.102 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.085+0000 7fac35125640 1 -- 172.21.15.169:0/4133948047 learned_addr learned my addr 172.21.15.169:0/4133948047 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:44:28.102 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.085+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3498513490 0 0) 0x7fac301a4160 con 0x7fac301027b0 2024-09-16T08:44:28.102 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.085+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac1c003520 con 0x7fac301027b0 2024-09-16T08:44:28.102 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.085+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 237098393 0 0) 0x7fac301a2f80 con 0x7fac3010a8c0 2024-09-16T08:44:28.102 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.085+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac301a4160 con 0x7fac3010a8c0 2024-09-16T08:44:28.102 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.085+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 600709061 0 0) 0x7fac301a5340 con 0x7fac30106c30 2024-09-16T08:44:28.102 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.085+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac301a2f80 con 0x7fac30106c30 2024-09-16T08:44:28.102 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.085+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1630489361 0 0) 0x7fac1c003520 con 0x7fac301027b0 2024-09-16T08:44:28.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.085+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fac301a5340 con 0x7fac301027b0 2024-09-16T08:44:28.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.085+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fac2c0019f0 con 0x7fac301027b0 2024-09-16T08:44:28.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.086+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 <== mon.1 v1:172.21.15.169:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1584145326 0 0) 0x7fac301a5340 con 0x7fac301027b0 2024-09-16T08:44:28.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.086+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 >> v1:172.21.15.160:6790/0 conn(0x7fac30106c30 legacy=0x7fac3019e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:44:28.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.086+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 >> v1:172.21.15.160:6789/0 conn(0x7fac3010a8c0 legacy=0x7fac301a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:44:28.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.086+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac301a6520 con 0x7fac301027b0 2024-09-16T08:44:28.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.086+0000 7fac37389640 1 -- 172.21.15.169:0/4133948047 --> v1:172.21.15.169:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fac301a5570 con 0x7fac301027b0 2024-09-16T08:44:28.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.086+0000 7fac37389640 1 -- 172.21.15.169:0/4133948047 --> v1:172.21.15.169:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fac301a5ba0 con 0x7fac301027b0 2024-09-16T08:44:28.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.086+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 <== mon.1 v1:172.21.15.169:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fac2c001c70 con 0x7fac301027b0 2024-09-16T08:44:28.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.086+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 <== mon.1 v1:172.21.15.169:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fac2c005c50 con 0x7fac301027b0 2024-09-16T08:44:28.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.087+0000 7fac37389640 1 -- 172.21.15.169:0/4133948047 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac140050b0 con 0x7fac301027b0 2024-09-16T08:44:28.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.087+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 <== mon.1 v1:172.21.15.169:6789/0 7 ==== mgrmap(e 15) ==== 103045+0+0 (unknown 4003825774 0 0) 0x7fac2c0038c0 con 0x7fac301027b0 2024-09-16T08:44:28.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.088+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 <== mon.1 v1:172.21.15.169:6789/0 8 ==== osd_map(40..40 src has 1..40) ==== 3497+0+0 (unknown 3398019081 0 0) 0x7fac2c0989c0 con 0x7fac301027b0 2024-09-16T08:44:28.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.092+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 <== mon.1 v1:172.21.15.169:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fac2c0606a0 con 0x7fac301027b0 2024-09-16T08:44:28.278 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:28.276+0000 7fac37389640 1 -- 172.21.15.169:0/4133948047 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fac14000bc0 con 0x7fac1c07b700 2024-09-16T08:44:28.937 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:28 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:44:28.937 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:28 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:44:28.937 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:28 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:28.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:28 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:44:28.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:28 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:44:28.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:28 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:28.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:28 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:44:28.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:28 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:44:28.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:28 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:29.935 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:29 smithi169 ceph-mon[33465]: pgmap v141: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:29.936 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:29 smithi169 ceph-mon[33465]: from='client.24286 v1:172.21.15.169:0/4133948047' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:44:29.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:29 smithi160 ceph-mon[30137]: pgmap v141: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:29.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:29 smithi160 ceph-mon[30137]: from='client.24286 v1:172.21.15.169:0/4133948047' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:44:29.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:29 smithi160 ceph-mon[40028]: pgmap v141: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:29.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:29 smithi160 ceph-mon[40028]: from='client.24286 v1:172.21.15.169:0/4133948047' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:44:30.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.169:0/3899340881' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3a03867-0d98-41bb-bbd7-cd6fb4f67eb2"} : dispatch 2024-09-16T08:44:30.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:30 smithi169 ceph-mon[33465]: from='client.24292 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3a03867-0d98-41bb-bbd7-cd6fb4f67eb2"} : dispatch 2024-09-16T08:44:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:30 smithi169 ceph-mon[33465]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3a03867-0d98-41bb-bbd7-cd6fb4f67eb2"}]': finished 2024-09-16T08:44:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:30 smithi169 ceph-mon[33465]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T08:44:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:44:30.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.169:0/3899340881' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3a03867-0d98-41bb-bbd7-cd6fb4f67eb2"} : dispatch 2024-09-16T08:44:30.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:30 smithi160 ceph-mon[30137]: from='client.24292 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3a03867-0d98-41bb-bbd7-cd6fb4f67eb2"} : dispatch 2024-09-16T08:44:30.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:30 smithi160 ceph-mon[30137]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3a03867-0d98-41bb-bbd7-cd6fb4f67eb2"}]': finished 2024-09-16T08:44:30.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:30 smithi160 ceph-mon[30137]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T08:44:30.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:44:30.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.169:0/3899340881' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3a03867-0d98-41bb-bbd7-cd6fb4f67eb2"} : dispatch 2024-09-16T08:44:30.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:30 smithi160 ceph-mon[40028]: from='client.24292 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3a03867-0d98-41bb-bbd7-cd6fb4f67eb2"} : dispatch 2024-09-16T08:44:30.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:30 smithi160 ceph-mon[40028]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3a03867-0d98-41bb-bbd7-cd6fb4f67eb2"}]': finished 2024-09-16T08:44:30.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:30 smithi160 ceph-mon[40028]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T08:44:30.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:44:31.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:31 smithi169 ceph-mon[33465]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:31.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.169:0/3148519886' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:44:31.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:31 smithi160 ceph-mon[30137]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:31.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.169:0/3148519886' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:44:31.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:31 smithi160 ceph-mon[40028]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:31.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.169:0/3148519886' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:44:33.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:33 smithi160 ceph-mon[30137]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-16T08:44:33.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:33 smithi160 ceph-mon[40028]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-16T08:44:34.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:33 smithi169 ceph-mon[33465]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-16T08:44:35.953 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:35 smithi169 ceph-mon[33465]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T08:44:36.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:35 smithi160 ceph-mon[30137]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T08:44:36.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:35 smithi160 ceph-mon[40028]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T08:44:38.029 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:37 smithi169 ceph-mon[33465]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-16T08:44:38.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:37 smithi160 ceph-mon[30137]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-16T08:44:38.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:37 smithi160 ceph-mon[40028]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-16T08:44:39.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:38 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-16T08:44:39.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:38 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:39.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:38 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-16T08:44:39.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:38 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:39.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:38 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-16T08:44:39.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:38 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:39.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:39 smithi169 ceph-mon[33465]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:44:39.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:39 smithi169 ceph-mon[33465]: Deploying daemon osd.6 on smithi169 2024-09-16T08:44:40.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:39 smithi160 ceph-mon[30137]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:44:40.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:39 smithi160 ceph-mon[30137]: Deploying daemon osd.6 on smithi169 2024-09-16T08:44:40.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:39 smithi160 ceph-mon[40028]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:44:40.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:39 smithi160 ceph-mon[40028]: Deploying daemon osd.6 on smithi169 2024-09-16T08:44:42.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:41 smithi169 ceph-mon[33465]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:42.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:41 smithi160 ceph-mon[30137]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:42.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:41 smithi160 ceph-mon[40028]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:44.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:43 smithi169 ceph-mon[33465]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:44.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:43 smithi160 ceph-mon[40028]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:44.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:43 smithi160 ceph-mon[30137]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:45.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:45 smithi169 ceph-mon[33465]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:45.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:45 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:44:45.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:45 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:45.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:45 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:46.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:45 smithi160 ceph-mon[30137]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:46.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:45 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:44:46.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:45 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:46.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:45 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:46.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:45 smithi160 ceph-mon[40028]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:46.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:45 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:44:46.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:45 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:46.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:45 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:48.028 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:47 smithi169 ceph-mon[33465]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:48.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:47 smithi160 ceph-mon[30137]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:48.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:47 smithi160 ceph-mon[40028]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:49.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:49 smithi169 ceph-mon[33465]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:49.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:49 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:49.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:49 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:49.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:49 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:49 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:44:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:49 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:49.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:49 smithi160 ceph-mon[30137]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:49.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:49 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:49.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:49 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:49 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:49 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:44:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:49 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:49 smithi160 ceph-mon[40028]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:49 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:49 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:49 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:49 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:44:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:49 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:51.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:51.342+0000 7fac267fc640 1 -- 172.21.15.169:0/4133948047 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 2957322077) 0x7fac14000bc0 con 0x7fac1c07b700 2024-09-16T08:44:51.344 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 6 on host 'smithi169' 2024-09-16T08:44:51.346 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:51.345+0000 7fac37389640 1 -- 172.21.15.169:0/4133948047 >> v1:172.21.15.160:6800/1286008384 conn(0x7fac1c07b700 legacy=0x7fac1c07dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:44:51.346 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:51.345+0000 7fac37389640 1 -- 172.21.15.169:0/4133948047 >> v1:172.21.15.169:6789/0 conn(0x7fac301027b0 legacy=0x7fac30197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:44:51.346 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:51.345+0000 7fac37389640 1 -- 172.21.15.169:0/4133948047 shutdown_connections 2024-09-16T08:44:51.346 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:51.345+0000 7fac37389640 1 -- 172.21.15.169:0/4133948047 >> 172.21.15.169:0/4133948047 conn(0x7fac300fe0a0 msgr2=0x7fac300fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:44:51.346 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:51.345+0000 7fac37389640 1 -- 172.21.15.169:0/4133948047 shutdown_connections 2024-09-16T08:44:51.346 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:51.345+0000 7fac37389640 1 -- 172.21.15.169:0/4133948047 wait complete. 2024-09-16T08:44:51.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:51 smithi169 ceph-mon[33465]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:51.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:51 smithi169 ceph-mon[33465]: from='osd.6 v1:172.21.15.169:6808/3462109725' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-16T08:44:51.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:51 smithi169 ceph-mon[33465]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-16T08:44:51.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:51 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:44:51.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:51 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:51.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:51 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:51.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:51 smithi160 ceph-mon[30137]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:51.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:51 smithi160 ceph-mon[30137]: from='osd.6 v1:172.21.15.169:6808/3462109725' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-16T08:44:51.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:51 smithi160 ceph-mon[30137]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-16T08:44:51.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:51 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:44:51.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:51 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:51.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:51 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:51.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:51 smithi160 ceph-mon[40028]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:51.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:51 smithi160 ceph-mon[40028]: from='osd.6 v1:172.21.15.169:6808/3462109725' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-16T08:44:51.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:51 smithi160 ceph-mon[40028]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-16T08:44:51.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:51 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:44:51.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:51 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:51.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:51 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:51.918 DEBUG:teuthology.orchestra.run.smithi169:osd.6> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.6.service 2024-09-16T08:44:51.924 INFO:tasks.cephadm:Deploying osd.7 on smithi169 with /dev/vg_nvme/lv_1... 2024-09-16T08:44:51.924 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T08:44:52.587 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:44:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:52 smithi169 ceph-mon[33465]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T08:44:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:52 smithi169 ceph-mon[33465]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T08:44:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:52 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:44:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:52 smithi169 ceph-mon[33465]: from='osd.6 v1:172.21.15.169:6808/3462109725' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:44:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:52 smithi169 ceph-mon[33465]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:44:52.600 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 08:44:52 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T08:44:52.361+0000 7ff8fa075640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T08:44:52.600 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 08:44:52 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T08:44:52.368+0000 7ff8f5e9c640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:44:52.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:52 smithi160 ceph-mon[30137]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T08:44:52.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:52 smithi160 ceph-mon[30137]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T08:44:52.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:52 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:44:52.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:52 smithi160 ceph-mon[30137]: from='osd.6 v1:172.21.15.169:6808/3462109725' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:44:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:52 smithi160 ceph-mon[30137]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:44:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:52 smithi160 ceph-mon[40028]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T08:44:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:52 smithi160 ceph-mon[40028]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T08:44:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:52 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:44:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:52 smithi160 ceph-mon[40028]: from='osd.6 v1:172.21.15.169:6808/3462109725' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:44:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:52 smithi160 ceph-mon[40028]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:44:53.495 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:53 smithi169 ceph-mon[33465]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:53.495 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:53 smithi169 ceph-mon[33465]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T08:44:53.495 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:53 smithi169 ceph-mon[33465]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T08:44:53.495 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:53 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:44:53.495 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:53 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:44:53.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:53 smithi160 ceph-mon[30137]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:53.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:53 smithi160 ceph-mon[30137]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T08:44:53.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:53 smithi160 ceph-mon[30137]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T08:44:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:53 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:44:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:53 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:44:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:53 smithi160 ceph-mon[40028]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:44:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:53 smithi160 ceph-mon[40028]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T08:44:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:53 smithi160 ceph-mon[40028]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T08:44:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:53 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:44:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:53 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:44:54.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:54 smithi169 ceph-mon[33465]: purged_snaps scrub starts 2024-09-16T08:44:54.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:54 smithi169 ceph-mon[33465]: purged_snaps scrub ok 2024-09-16T08:44:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:54 smithi169 ceph-mon[33465]: osd.6 v1:172.21.15.169:6808/3462109725 boot 2024-09-16T08:44:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:54 smithi169 ceph-mon[33465]: osdmap e44: 7 total, 7 up, 7 in 2024-09-16T08:44:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:54 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:44:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:54 smithi169 ceph-mon[33465]: osdmap e45: 7 total, 7 up, 7 in 2024-09-16T08:44:54.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:54 smithi160 ceph-mon[30137]: purged_snaps scrub starts 2024-09-16T08:44:54.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:54 smithi160 ceph-mon[30137]: purged_snaps scrub ok 2024-09-16T08:44:54.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:54 smithi160 ceph-mon[30137]: osd.6 v1:172.21.15.169:6808/3462109725 boot 2024-09-16T08:44:54.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:54 smithi160 ceph-mon[30137]: osdmap e44: 7 total, 7 up, 7 in 2024-09-16T08:44:54.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:54 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:44:54.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:54 smithi160 ceph-mon[30137]: osdmap e45: 7 total, 7 up, 7 in 2024-09-16T08:44:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:54 smithi160 ceph-mon[40028]: purged_snaps scrub starts 2024-09-16T08:44:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:54 smithi160 ceph-mon[40028]: purged_snaps scrub ok 2024-09-16T08:44:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:54 smithi160 ceph-mon[40028]: osd.6 v1:172.21.15.169:6808/3462109725 boot 2024-09-16T08:44:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:54 smithi160 ceph-mon[40028]: osdmap e44: 7 total, 7 up, 7 in 2024-09-16T08:44:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:54 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:44:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:54 smithi160 ceph-mon[40028]: osdmap e45: 7 total, 7 up, 7 in 2024-09-16T08:44:55.580 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:55 smithi169 ceph-mon[33465]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:44:55.580 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:55 smithi169 ceph-mon[33465]: osdmap e46: 7 total, 7 up, 7 in 2024-09-16T08:44:55.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:55 smithi160 ceph-mon[30137]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:44:55.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:55 smithi160 ceph-mon[30137]: osdmap e46: 7 total, 7 up, 7 in 2024-09-16T08:44:55.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:55 smithi160 ceph-mon[40028]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:44:55.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:55 smithi160 ceph-mon[40028]: osdmap e46: 7 total, 7 up, 7 in 2024-09-16T08:44:56.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[30137]: Detected new or changed devices on smithi169 2024-09-16T08:44:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:44:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T08:44:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T08:44:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[30137]: Adjusting osd_memory_target on smithi169 to 5696M 2024-09-16T08:44:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:44:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:56.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[40028]: Detected new or changed devices on smithi169 2024-09-16T08:44:56.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:56.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:56.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:44:56.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T08:44:56.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T08:44:56.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[40028]: Adjusting osd_memory_target on smithi169 to 5696M 2024-09-16T08:44:56.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:56.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:56.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:44:56.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:56.987 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:56 smithi169 ceph-mon[33465]: Detected new or changed devices on smithi169 2024-09-16T08:44:56.987 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:56.987 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:56.987 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:44:56.987 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T08:44:56.987 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T08:44:56.988 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:56 smithi169 ceph-mon[33465]: Adjusting osd_memory_target on smithi169 to 5696M 2024-09-16T08:44:56.988 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:56.988 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:56.988 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:44:56.988 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:44:57.258 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T08:44:57.284 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_1 2024-09-16T08:44:57.532 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:44:57.916 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:57 smithi169 ceph-mon[33465]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:44:57.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:57 smithi160 ceph-mon[30137]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:44:57.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:57 smithi160 ceph-mon[40028]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:44:58.024 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.000+0000 7ffbe24a1640 1 -- 172.21.15.169:0/2043630398 >> v1:172.21.15.169:6789/0 conn(0x7ffbdc1016a0 legacy=0x7ffbdc101aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:44:58.024 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.001+0000 7ffbe24a1640 1 -- 172.21.15.169:0/2043630398 shutdown_connections 2024-09-16T08:44:58.024 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.001+0000 7ffbe24a1640 1 -- 172.21.15.169:0/2043630398 >> 172.21.15.169:0/2043630398 conn(0x7ffbdc0fcf70 msgr2=0x7ffbdc0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:44:58.024 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.001+0000 7ffbe24a1640 1 -- 172.21.15.169:0/2043630398 shutdown_connections 2024-09-16T08:44:58.024 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.001+0000 7ffbe24a1640 1 -- 172.21.15.169:0/2043630398 wait complete. 2024-09-16T08:44:58.025 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.001+0000 7ffbe24a1640 1 Processor -- start 2024-09-16T08:44:58.025 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.001+0000 7ffbe24a1640 1 -- start start 2024-09-16T08:44:58.025 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.001+0000 7ffbe24a1640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ffbdc1a1e10 con 0x7ffbdc105b20 2024-09-16T08:44:58.025 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.001+0000 7ffbe24a1640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ffbdc1a2ff0 con 0x7ffbdc1097b0 2024-09-16T08:44:58.025 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbe24a1640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ffbdc1a41d0 con 0x7ffbdc1016a0 2024-09-16T08:44:58.025 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbe0a3e640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7ffbdc1097b0 0x7ffbdc1a06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:56360/0 (socket says 172.21.15.169:56360) 2024-09-16T08:44:58.025 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbe0a3e640 1 -- 172.21.15.169:0/2183336842 learned_addr learned my addr 172.21.15.169:0/2183336842 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:44:58.025 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2934241735 0 0) 0x7ffbdc1a2ff0 con 0x7ffbdc1097b0 2024-09-16T08:44:58.025 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffbc4003520 con 0x7ffbdc1097b0 2024-09-16T08:44:58.025 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1688602603 0 0) 0x7ffbdc1a41d0 con 0x7ffbdc1016a0 2024-09-16T08:44:58.025 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffbdc1a2ff0 con 0x7ffbdc1016a0 2024-09-16T08:44:58.025 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 603827946 0 0) 0x7ffbdc1a1e10 con 0x7ffbdc105b20 2024-09-16T08:44:58.026 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffbdc1a41d0 con 0x7ffbdc105b20 2024-09-16T08:44:58.026 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2785928439 0 0) 0x7ffbc4003520 con 0x7ffbdc1097b0 2024-09-16T08:44:58.026 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffbdc1a1e10 con 0x7ffbdc1097b0 2024-09-16T08:44:58.026 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ffbd4001d30 con 0x7ffbdc1097b0 2024-09-16T08:44:58.026 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1043891648 0 0) 0x7ffbdc1a41d0 con 0x7ffbdc105b20 2024-09-16T08:44:58.026 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffbc4003520 con 0x7ffbdc105b20 2024-09-16T08:44:58.026 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ffbd0001760 con 0x7ffbdc105b20 2024-09-16T08:44:58.026 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3856027288 0 0) 0x7ffbdc1a2ff0 con 0x7ffbdc1016a0 2024-09-16T08:44:58.026 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffbdc1a41d0 con 0x7ffbdc1016a0 2024-09-16T08:44:58.026 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ffbcc0014b0 con 0x7ffbdc1016a0 2024-09-16T08:44:58.026 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 <== mon.1 v1:172.21.15.169:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1098896732 0 0) 0x7ffbdc1a1e10 con 0x7ffbdc1097b0 2024-09-16T08:44:58.026 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 >> v1:172.21.15.160:6790/0 conn(0x7ffbdc1016a0 legacy=0x7ffbdc196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:44:58.026 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 >> v1:172.21.15.160:6789/0 conn(0x7ffbdc105b20 legacy=0x7ffbdc19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:44:58.027 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbdc1a53b0 con 0x7ffbdc1097b0 2024-09-16T08:44:58.027 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbe24a1640 1 -- 172.21.15.169:0/2183336842 --> v1:172.21.15.169:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ffbdc1a4400 con 0x7ffbdc1097b0 2024-09-16T08:44:58.027 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbe24a1640 1 -- 172.21.15.169:0/2183336842 --> v1:172.21.15.169:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ffbdc1a4a30 con 0x7ffbdc1097b0 2024-09-16T08:44:58.027 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 <== mon.1 v1:172.21.15.169:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7ffbd4003e00 con 0x7ffbdc1097b0 2024-09-16T08:44:58.027 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.002+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 <== mon.1 v1:172.21.15.169:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ffbd4005eb0 con 0x7ffbdc1097b0 2024-09-16T08:44:58.027 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.003+0000 7ffbe24a1640 1 -- 172.21.15.169:0/2183336842 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc80050b0 con 0x7ffbdc1097b0 2024-09-16T08:44:58.027 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.004+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 <== mon.1 v1:172.21.15.169:6789/0 7 ==== mgrmap(e 15) ==== 103045+0+0 (unknown 4003825774 0 0) 0x7ffbd4003990 con 0x7ffbdc1097b0 2024-09-16T08:44:58.027 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.005+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 <== mon.1 v1:172.21.15.169:6789/0 8 ==== osd_map(46..46 src has 1..46) ==== 3789+0+0 (unknown 2616762976 0 0) 0x7ffbd4098a90 con 0x7ffbdc1097b0 2024-09-16T08:44:58.027 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.009+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 <== mon.1 v1:172.21.15.169:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7ffbd4060610 con 0x7ffbdc1097b0 2024-09-16T08:44:58.194 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:44:58.193+0000 7ffbe24a1640 1 -- 172.21.15.169:0/2183336842 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ffbc8000bc0 con 0x7ffbc407bc00 2024-09-16T08:44:58.827 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:58 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:44:58.827 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:58 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:44:58.827 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:58 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:58.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:58 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:44:58.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:58 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:44:58.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:58 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:58.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:58 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T08:44:58.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:58 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T08:44:58.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:58 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:44:59.785 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:59 smithi169 ceph-mon[33465]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:44:59.786 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:44:59 smithi169 ceph-mon[33465]: from='client.24313 v1:172.21.15.169:0/2183336842' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:44:59.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:59 smithi160 ceph-mon[30137]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:44:59.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:44:59 smithi160 ceph-mon[30137]: from='client.24313 v1:172.21.15.169:0/2183336842' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:44:59.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:59 smithi160 ceph-mon[40028]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:44:59.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:44:59 smithi160 ceph-mon[40028]: from='client.24313 v1:172.21.15.169:0/2183336842' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:00.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.169:0/4262198260' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a24f20d7-85f2-4c15-bab9-05382f876001"} : dispatch 2024-09-16T08:45:00.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:00 smithi160 ceph-mon[30137]: from='client.24319 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a24f20d7-85f2-4c15-bab9-05382f876001"} : dispatch 2024-09-16T08:45:00.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:00 smithi160 ceph-mon[30137]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a24f20d7-85f2-4c15-bab9-05382f876001"}]': finished 2024-09-16T08:45:00.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:00 smithi160 ceph-mon[30137]: osdmap e47: 8 total, 7 up, 8 in 2024-09-16T08:45:00.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:00 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T08:45:00.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.169:0/4262198260' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a24f20d7-85f2-4c15-bab9-05382f876001"} : dispatch 2024-09-16T08:45:00.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:00 smithi160 ceph-mon[40028]: from='client.24319 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a24f20d7-85f2-4c15-bab9-05382f876001"} : dispatch 2024-09-16T08:45:00.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:00 smithi160 ceph-mon[40028]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a24f20d7-85f2-4c15-bab9-05382f876001"}]': finished 2024-09-16T08:45:00.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:00 smithi160 ceph-mon[40028]: osdmap e47: 8 total, 7 up, 8 in 2024-09-16T08:45:00.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:00 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T08:45:01.043 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.169:0/4262198260' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a24f20d7-85f2-4c15-bab9-05382f876001"} : dispatch 2024-09-16T08:45:01.043 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:00 smithi169 ceph-mon[33465]: from='client.24319 ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a24f20d7-85f2-4c15-bab9-05382f876001"} : dispatch 2024-09-16T08:45:01.043 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:00 smithi169 ceph-mon[33465]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a24f20d7-85f2-4c15-bab9-05382f876001"}]': finished 2024-09-16T08:45:01.043 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:00 smithi169 ceph-mon[33465]: osdmap e47: 8 total, 7 up, 8 in 2024-09-16T08:45:01.043 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:00 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T08:45:01.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:01 smithi160 ceph-mon[30137]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:01.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.169:0/1376885781' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:45:01.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:01 smithi160 ceph-mon[40028]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:01.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.169:0/1376885781' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:45:02.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:01 smithi169 ceph-mon[33465]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:02.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.169:0/1376885781' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T08:45:03.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:03 smithi160 ceph-mon[30137]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:03.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:03 smithi160 ceph-mon[40028]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:04.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:03 smithi169 ceph-mon[33465]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:05.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:05 smithi160 ceph-mon[30137]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:05.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:05 smithi160 ceph-mon[40028]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:06.009 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:05 smithi169 ceph-mon[33465]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:07.803 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:07 smithi169 ceph-mon[33465]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:07.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:07 smithi160 ceph-mon[30137]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:07.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:07 smithi160 ceph-mon[40028]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:08 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-16T08:45:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:08 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:08.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:08 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-16T08:45:08.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:08 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:08 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-16T08:45:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:08 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:09.812 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:09 smithi169 ceph-mon[33465]: Deploying daemon osd.7 on smithi169 2024-09-16T08:45:09.812 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:09 smithi169 ceph-mon[33465]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:09.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:09 smithi160 ceph-mon[30137]: Deploying daemon osd.7 on smithi169 2024-09-16T08:45:09.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:09 smithi160 ceph-mon[30137]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:09.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:09 smithi160 ceph-mon[40028]: Deploying daemon osd.7 on smithi169 2024-09-16T08:45:09.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:09 smithi160 ceph-mon[40028]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:11.831 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:11 smithi169 ceph-mon[33465]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:11.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:11 smithi160 ceph-mon[30137]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:11.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:11 smithi160 ceph-mon[40028]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:13.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:13 smithi160 ceph-mon[30137]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:13.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:13 smithi160 ceph-mon[40028]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:14.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:13 smithi169 ceph-mon[33465]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:15.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:15 smithi160 ceph-mon[40028]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:15.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:15 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:45:15.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:15 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:15.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:15 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:15.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:15 smithi160 ceph-mon[30137]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:15.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:15 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:45:15.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:15 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:15.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:15 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:16.039 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:15 smithi169 ceph-mon[33465]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:16.039 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:15 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:45:16.039 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:15 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:16.039 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:15 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:17.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:17 smithi169 ceph-mon[33465]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:17.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:17 smithi160 ceph-mon[30137]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:17.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:17 smithi160 ceph-mon[40028]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:19.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:19 smithi160 ceph-mon[30137]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:19.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:19 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:19 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:19 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:19 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:45:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:19 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:19.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:19 smithi160 ceph-mon[40028]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:19.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:19 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:19.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:19 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:19.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:19 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:19.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:19 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:45:19.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:19 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:19.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:19 smithi169 ceph-mon[33465]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:19 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:19 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:19 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:19 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:45:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:19 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:20.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:20.908+0000 7ffbd97fa640 1 -- 172.21.15.169:0/2183336842 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 3964717545) 0x7ffbc8000bc0 con 0x7ffbc407bc00 2024-09-16T08:45:20.909 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 7 on host 'smithi169' 2024-09-16T08:45:20.911 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:20.911+0000 7ffbe24a1640 1 -- 172.21.15.169:0/2183336842 >> v1:172.21.15.160:6800/1286008384 conn(0x7ffbc407bc00 legacy=0x7ffbc407e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:20.911 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:20.911+0000 7ffbe24a1640 1 -- 172.21.15.169:0/2183336842 >> v1:172.21.15.169:6789/0 conn(0x7ffbdc1097b0 legacy=0x7ffbdc1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:20.911 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:20.911+0000 7ffbe24a1640 1 -- 172.21.15.169:0/2183336842 shutdown_connections 2024-09-16T08:45:20.912 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:20.911+0000 7ffbe24a1640 1 -- 172.21.15.169:0/2183336842 >> 172.21.15.169:0/2183336842 conn(0x7ffbdc0fcf70 msgr2=0x7ffbdc109c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:20.912 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:20.911+0000 7ffbe24a1640 1 -- 172.21.15.169:0/2183336842 shutdown_connections 2024-09-16T08:45:20.912 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:20.911+0000 7ffbe24a1640 1 -- 172.21.15.169:0/2183336842 wait complete. 2024-09-16T08:45:21.266 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:21 smithi169 ceph-mon[33465]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:21.266 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:21 smithi169 ceph-mon[33465]: from='osd.7 v1:172.21.15.169:6812/1196718305' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-16T08:45:21.266 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:21 smithi169 ceph-mon[33465]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-16T08:45:21.266 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:21 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:45:21.266 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:21 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:21.266 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:21 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:21.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:21 smithi160 ceph-mon[30137]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:21.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:21 smithi160 ceph-mon[30137]: from='osd.7 v1:172.21.15.169:6812/1196718305' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-16T08:45:21.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:21 smithi160 ceph-mon[30137]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-16T08:45:21.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:21 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:45:21.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:21 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:21.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:21 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:21.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:21 smithi160 ceph-mon[40028]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:21.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:21 smithi160 ceph-mon[40028]: from='osd.7 v1:172.21.15.169:6812/1196718305' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-16T08:45:21.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:21 smithi160 ceph-mon[40028]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-16T08:45:21.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:21 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:45:21.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:21 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:21.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:21 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:21.550 DEBUG:teuthology.orchestra.run.smithi169:osd.7> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.7.service 2024-09-16T08:45:21.553 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-16T08:45:21.554 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T08:45:21.810 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:22.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:22 smithi160 ceph-mon[30137]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T08:45:22.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:22 smithi160 ceph-mon[30137]: from='osd.7 v1:172.21.15.169:6812/1196718305' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:45:22.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:22 smithi160 ceph-mon[30137]: osdmap e48: 8 total, 7 up, 8 in 2024-09-16T08:45:22.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:22 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T08:45:22.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:22 smithi160 ceph-mon[30137]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:45:22.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:22 smithi160 ceph-mon[40028]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T08:45:22.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:22 smithi160 ceph-mon[40028]: from='osd.7 v1:172.21.15.169:6812/1196718305' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:45:22.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:22 smithi160 ceph-mon[40028]: osdmap e48: 8 total, 7 up, 8 in 2024-09-16T08:45:22.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:22 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T08:45:22.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:22 smithi160 ceph-mon[40028]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:45:22.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.205+0000 7fb0a22dd640 1 -- 172.21.15.160:0/3258647399 >> v1:172.21.15.160:6790/0 conn(0x7fb09c1027b0 legacy=0x7fb09c102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:22.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.205+0000 7fb0a22dd640 1 -- 172.21.15.160:0/3258647399 shutdown_connections 2024-09-16T08:45:22.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.205+0000 7fb0a22dd640 1 -- 172.21.15.160:0/3258647399 >> 172.21.15.160:0/3258647399 conn(0x7fb09c0fe0a0 msgr2=0x7fb09c1004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:22.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.205+0000 7fb0a22dd640 1 -- 172.21.15.160:0/3258647399 shutdown_connections 2024-09-16T08:45:22.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.205+0000 7fb0a22dd640 1 -- 172.21.15.160:0/3258647399 wait complete. 2024-09-16T08:45:22.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.205+0000 7fb0a22dd640 1 Processor -- start 2024-09-16T08:45:22.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.206+0000 7fb0a22dd640 1 -- start start 2024-09-16T08:45:22.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.206+0000 7fb0a22dd640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb09c1a2db0 con 0x7fb09c1027b0 2024-09-16T08:45:22.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.206+0000 7fb0a22dd640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb09c1a3f90 con 0x7fb09c106c30 2024-09-16T08:45:22.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.206+0000 7fb0a22dd640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb09c1a5170 con 0x7fb09c10a8c0 2024-09-16T08:45:22.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.206+0000 7fb09bfff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fb09c1027b0 0x7fb09c197d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:37312/0 (socket says 172.21.15.160:37312) 2024-09-16T08:45:22.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.206+0000 7fb09bfff640 1 -- 172.21.15.160:0/3809932626 learned_addr learned my addr 172.21.15.160:0/3809932626 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:45:22.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.206+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 88967055 0 0) 0x7fb09c1a2db0 con 0x7fb09c1027b0 2024-09-16T08:45:22.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.206+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb084003520 con 0x7fb09c1027b0 2024-09-16T08:45:22.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.206+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 790222301 0 0) 0x7fb09c1a5170 con 0x7fb09c10a8c0 2024-09-16T08:45:22.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.206+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb09c1a2db0 con 0x7fb09c10a8c0 2024-09-16T08:45:22.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.206+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2474973583 0 0) 0x7fb09c1a3f90 con 0x7fb09c106c30 2024-09-16T08:45:22.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.206+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb09c1a5170 con 0x7fb09c106c30 2024-09-16T08:45:22.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.206+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2375807144 0 0) 0x7fb084003520 con 0x7fb09c1027b0 2024-09-16T08:45:22.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.206+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb09c1a3f90 con 0x7fb09c1027b0 2024-09-16T08:45:22.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.206+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb08c001430 con 0x7fb09c1027b0 2024-09-16T08:45:22.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.207+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 381419632 0 0) 0x7fb09c1a2db0 con 0x7fb09c10a8c0 2024-09-16T08:45:22.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.207+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb084003520 con 0x7fb09c10a8c0 2024-09-16T08:45:22.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.207+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb094001bf0 con 0x7fb09c10a8c0 2024-09-16T08:45:22.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.207+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1480648487 0 0) 0x7fb09c1a5170 con 0x7fb09c106c30 2024-09-16T08:45:22.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.207+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb09c1a2db0 con 0x7fb09c106c30 2024-09-16T08:45:22.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.207+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb0900013b0 con 0x7fb09c106c30 2024-09-16T08:45:22.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.207+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4232456364 0 0) 0x7fb09c1a3f90 con 0x7fb09c1027b0 2024-09-16T08:45:22.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.207+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 >> v1:172.21.15.160:6790/0 conn(0x7fb09c10a8c0 legacy=0x7fb09c1a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:22.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.207+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 >> v1:172.21.15.169:6789/0 conn(0x7fb09c106c30 legacy=0x7fb09c19df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:22.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.207+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09c1a6350 con 0x7fb09c1027b0 2024-09-16T08:45:22.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.207+0000 7fb0a22dd640 1 -- 172.21.15.160:0/3809932626 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb09c1a2fe0 con 0x7fb09c1027b0 2024-09-16T08:45:22.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.207+0000 7fb0a22dd640 1 -- 172.21.15.160:0/3809932626 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb09c1a3610 con 0x7fb09c1027b0 2024-09-16T08:45:22.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.207+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fb08c0035f0 con 0x7fb09c1027b0 2024-09-16T08:45:22.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.207+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb08c004c40 con 0x7fb09c1027b0 2024-09-16T08:45:22.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.208+0000 7fb0a22dd640 1 -- 172.21.15.160:0/3809932626 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0880050b0 con 0x7fb09c1027b0 2024-09-16T08:45:22.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.208+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 15) ==== 103045+0+0 (unknown 4003825774 0 0) 0x7fb08c004c40 con 0x7fb09c1027b0 2024-09-16T08:45:22.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.209+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(49..49 src has 1..49) ==== 3941+0+0 (unknown 2895692782 0 0) 0x7fb08c098a40 con 0x7fb09c1027b0 2024-09-16T08:45:22.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.213+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fb08c060580 con 0x7fb09c1027b0 2024-09-16T08:45:22.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.388+0000 7fb0a22dd640 1 -- 172.21.15.160:0/3809932626 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb0880053a0 con 0x7fb09c1027b0 2024-09-16T08:45:22.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.389+0000 7fb0997fa640 1 -- 172.21.15.160:0/3809932626 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v49) ==== 74+0+130 (unknown 129372473 0 2033778154) 0x7fb08c064960 con 0x7fb09c1027b0 2024-09-16T08:45:22.390 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:45:22.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.391+0000 7fb0a22dd640 1 -- 172.21.15.160:0/3809932626 >> v1:172.21.15.160:6800/1286008384 conn(0x7fb08407bc70 legacy=0x7fb08407e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:22.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.391+0000 7fb0a22dd640 1 -- 172.21.15.160:0/3809932626 >> v1:172.21.15.160:6789/0 conn(0x7fb09c1027b0 legacy=0x7fb09c197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:22.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.391+0000 7fb0a22dd640 1 -- 172.21.15.160:0/3809932626 shutdown_connections 2024-09-16T08:45:22.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.391+0000 7fb0a22dd640 1 -- 172.21.15.160:0/3809932626 >> 172.21.15.160:0/3809932626 conn(0x7fb09c0fe0a0 msgr2=0x7fb09c10cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:22.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.392+0000 7fb0a22dd640 1 -- 172.21.15.160:0/3809932626 shutdown_connections 2024-09-16T08:45:22.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:22.392+0000 7fb0a22dd640 1 -- 172.21.15.160:0/3809932626 wait complete. 2024-09-16T08:45:22.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:22 smithi169 ceph-mon[33465]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T08:45:22.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:22 smithi169 ceph-mon[33465]: from='osd.7 v1:172.21.15.169:6812/1196718305' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:45:22.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:22 smithi169 ceph-mon[33465]: osdmap e48: 8 total, 7 up, 8 in 2024-09-16T08:45:22.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:22 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T08:45:22.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:22 smithi169 ceph-mon[33465]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]} : dispatch 2024-09-16T08:45:22.738 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 08:45:22 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T08:45:22.570+0000 7f51f9382640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T08:45:22.739 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 08:45:22 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T08:45:22.577+0000 7f51f098d640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:45:23.094 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":49,"num_osds":8,"num_up_osds":7,"osd_up_since":1726476293,"num_in_osds":8,"osd_in_since":1726476300,"num_remapped_pgs":0} 2024-09-16T08:45:23.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:23 smithi169 ceph-mon[33465]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:23.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:23 smithi169 ceph-mon[33465]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T08:45:23.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:23 smithi169 ceph-mon[33465]: osdmap e49: 8 total, 7 up, 8 in 2024-09-16T08:45:23.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:23 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T08:45:23.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:23 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3809932626' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-16T08:45:23.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:23 smithi169 ceph-mon[33465]: from='osd.7 ' entity='osd.7' 2024-09-16T08:45:23.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:23 smithi160 ceph-mon[30137]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:23 smithi160 ceph-mon[30137]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T08:45:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:23 smithi160 ceph-mon[30137]: osdmap e49: 8 total, 7 up, 8 in 2024-09-16T08:45:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:23 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T08:45:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:23 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3809932626' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-16T08:45:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:23 smithi160 ceph-mon[30137]: from='osd.7 ' entity='osd.7' 2024-09-16T08:45:23.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:23 smithi160 ceph-mon[40028]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:45:23.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:23 smithi160 ceph-mon[40028]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T08:45:23.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:23 smithi160 ceph-mon[40028]: osdmap e49: 8 total, 7 up, 8 in 2024-09-16T08:45:23.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:23 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T08:45:23.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:23 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3809932626' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-16T08:45:23.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:23 smithi160 ceph-mon[40028]: from='osd.7 ' entity='osd.7' 2024-09-16T08:45:24.096 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T08:45:24.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:24 smithi169 ceph-mon[33465]: purged_snaps scrub starts 2024-09-16T08:45:24.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:24 smithi169 ceph-mon[33465]: purged_snaps scrub ok 2024-09-16T08:45:24.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:24 smithi169 ceph-mon[33465]: osd.7 v1:172.21.15.169:6812/1196718305 boot 2024-09-16T08:45:24.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:24 smithi169 ceph-mon[33465]: osdmap e50: 8 total, 8 up, 8 in 2024-09-16T08:45:24.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:24 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T08:45:24.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:24 smithi169 ceph-mon[33465]: osdmap e51: 8 total, 8 up, 8 in 2024-09-16T08:45:24.355 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:24.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:24 smithi160 ceph-mon[30137]: purged_snaps scrub starts 2024-09-16T08:45:24.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:24 smithi160 ceph-mon[30137]: purged_snaps scrub ok 2024-09-16T08:45:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:24 smithi160 ceph-mon[30137]: osd.7 v1:172.21.15.169:6812/1196718305 boot 2024-09-16T08:45:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:24 smithi160 ceph-mon[30137]: osdmap e50: 8 total, 8 up, 8 in 2024-09-16T08:45:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:24 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T08:45:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:24 smithi160 ceph-mon[30137]: osdmap e51: 8 total, 8 up, 8 in 2024-09-16T08:45:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:24 smithi160 ceph-mon[40028]: purged_snaps scrub starts 2024-09-16T08:45:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:24 smithi160 ceph-mon[40028]: purged_snaps scrub ok 2024-09-16T08:45:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:24 smithi160 ceph-mon[40028]: osd.7 v1:172.21.15.169:6812/1196718305 boot 2024-09-16T08:45:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:24 smithi160 ceph-mon[40028]: osdmap e50: 8 total, 8 up, 8 in 2024-09-16T08:45:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:24 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T08:45:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:24 smithi160 ceph-mon[40028]: osdmap e51: 8 total, 8 up, 8 in 2024-09-16T08:45:24.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.810+0000 7f73bebb8640 1 -- 172.21.15.160:0/1830471688 >> v1:172.21.15.160:6790/0 conn(0x7f73b810a8c0 legacy=0x7f73b810cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:24.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.810+0000 7f73bebb8640 1 -- 172.21.15.160:0/1830471688 shutdown_connections 2024-09-16T08:45:24.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.810+0000 7f73bebb8640 1 -- 172.21.15.160:0/1830471688 >> 172.21.15.160:0/1830471688 conn(0x7f73b80fe0a0 msgr2=0x7f73b81004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:24.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.810+0000 7f73bebb8640 1 -- 172.21.15.160:0/1830471688 shutdown_connections 2024-09-16T08:45:24.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.810+0000 7f73bebb8640 1 -- 172.21.15.160:0/1830471688 wait complete. 2024-09-16T08:45:24.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.811+0000 7f73bebb8640 1 Processor -- start 2024-09-16T08:45:24.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.811+0000 7f73bebb8640 1 -- start start 2024-09-16T08:45:24.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.811+0000 7f73bebb8640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f73b81a2e30 con 0x7f73b8106c30 2024-09-16T08:45:24.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.811+0000 7f73bebb8640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f73b81a4010 con 0x7f73b81027b0 2024-09-16T08:45:24.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.811+0000 7f73bebb8640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f73b81a51f0 con 0x7f73b810a8c0 2024-09-16T08:45:24.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.811+0000 7f73b7fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f73b8106c30 0x7f73b819dfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:37330/0 (socket says 172.21.15.160:37330) 2024-09-16T08:45:24.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.811+0000 7f73bd155640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f73b810a8c0 0x7f73b81a1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:49612/0 (socket says 172.21.15.160:49612) 2024-09-16T08:45:24.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.811+0000 7f73b7fff640 1 -- 172.21.15.160:0/99845980 learned_addr learned my addr 172.21.15.160:0/99845980 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:45:24.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.811+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2605760965 0 0) 0x7f73b81a51f0 con 0x7f73b810a8c0 2024-09-16T08:45:24.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.811+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73a8005020 con 0x7f73b810a8c0 2024-09-16T08:45:24.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.811+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 816014901 0 0) 0x7f73b81a2e30 con 0x7f73b8106c30 2024-09-16T08:45:24.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.811+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73b81a51f0 con 0x7f73b8106c30 2024-09-16T08:45:24.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.811+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3470457769 0 0) 0x7f73b81a4010 con 0x7f73b81027b0 2024-09-16T08:45:24.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.811+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73b81a2e30 con 0x7f73b81027b0 2024-09-16T08:45:24.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 536550316 0 0) 0x7f73b81a51f0 con 0x7f73b8106c30 2024-09-16T08:45:24.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f73b81a4010 con 0x7f73b8106c30 2024-09-16T08:45:24.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f73a40019b0 con 0x7f73b8106c30 2024-09-16T08:45:24.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3515723996 0 0) 0x7f73a8005020 con 0x7f73b810a8c0 2024-09-16T08:45:24.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f73b81a51f0 con 0x7f73b810a8c0 2024-09-16T08:45:24.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3608767491 0 0) 0x7f73b81a2e30 con 0x7f73b81027b0 2024-09-16T08:45:24.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f73a8005020 con 0x7f73b81027b0 2024-09-16T08:45:24.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f73b0003a00 con 0x7f73b810a8c0 2024-09-16T08:45:24.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f73ac001240 con 0x7f73b81027b0 2024-09-16T08:45:24.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 223578050 0 0) 0x7f73b81a4010 con 0x7f73b8106c30 2024-09-16T08:45:24.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 >> v1:172.21.15.160:6790/0 conn(0x7f73b810a8c0 legacy=0x7f73b81a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:24.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 >> v1:172.21.15.169:6789/0 conn(0x7f73b81027b0 legacy=0x7f73b8197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:24.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b81a63d0 con 0x7f73b8106c30 2024-09-16T08:45:24.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73bebb8640 1 -- 172.21.15.160:0/99845980 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f73b81a5420 con 0x7f73b8106c30 2024-09-16T08:45:24.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f73a4003a80 con 0x7f73b8106c30 2024-09-16T08:45:24.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f73a4005c20 con 0x7f73b8106c30 2024-09-16T08:45:24.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.812+0000 7f73bebb8640 1 -- 172.21.15.160:0/99845980 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f73b81a59d0 con 0x7f73b8106c30 2024-09-16T08:45:24.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.813+0000 7f73bebb8640 1 -- 172.21.15.160:0/99845980 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b810e7f0 con 0x7f73b8106c30 2024-09-16T08:45:24.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.814+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 15) ==== 103045+0+0 (unknown 4003825774 0 0) 0x7f73a4003610 con 0x7f73b8106c30 2024-09-16T08:45:24.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.814+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(51..51 src has 1..51) ==== 4081+0+0 (unknown 409745828 0 0) 0x7f73a4098b10 con 0x7f73b8106c30 2024-09-16T08:45:24.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.818+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f73a4060560 con 0x7f73b8106c30 2024-09-16T08:45:24.994 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.993+0000 7f73bebb8640 1 -- 172.21.15.160:0/99845980 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f73b81a5db0 con 0x7f73b8106c30 2024-09-16T08:45:24.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.993+0000 7f73b5ffb640 1 -- 172.21.15.160:0/99845980 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v51) ==== 74+0+130 (unknown 4287931526 0 2445126951) 0x7f73a4064940 con 0x7f73b8106c30 2024-09-16T08:45:24.995 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:45:24.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.996+0000 7f73bebb8640 1 -- 172.21.15.160:0/99845980 >> v1:172.21.15.160:6800/1286008384 conn(0x7f73a807bd20 legacy=0x7f73a807e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:24.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.996+0000 7f73bebb8640 1 -- 172.21.15.160:0/99845980 >> v1:172.21.15.160:6789/0 conn(0x7f73b8106c30 legacy=0x7f73b819dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:24.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.996+0000 7f73bebb8640 1 -- 172.21.15.160:0/99845980 shutdown_connections 2024-09-16T08:45:24.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.996+0000 7f73bebb8640 1 -- 172.21.15.160:0/99845980 >> 172.21.15.160:0/99845980 conn(0x7f73b80fe0a0 msgr2=0x7f73b8100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:24.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.996+0000 7f73bebb8640 1 -- 172.21.15.160:0/99845980 shutdown_connections 2024-09-16T08:45:24.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:24.996+0000 7f73bebb8640 1 -- 172.21.15.160:0/99845980 wait complete. 2024-09-16T08:45:25.695 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[40028]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:25.695 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[40028]: Detected new or changed devices on smithi169 2024-09-16T08:45:25.695 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:25.695 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:25.695 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:45:25.695 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T08:45:25.695 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T08:45:25.696 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T08:45:25.696 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[40028]: Adjusting osd_memory_target on smithi169 to 4272M 2024-09-16T08:45:25.696 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:25.696 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:25.696 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:45:25.696 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:25.696 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/99845980' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-16T08:45:25.696 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[40028]: osdmap e52: 8 total, 8 up, 8 in 2024-09-16T08:45:25.696 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[30137]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:25.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[30137]: Detected new or changed devices on smithi169 2024-09-16T08:45:25.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:25.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:25.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:45:25.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T08:45:25.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T08:45:25.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T08:45:25.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[30137]: Adjusting osd_memory_target on smithi169 to 4272M 2024-09-16T08:45:25.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:25.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:25.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:45:25.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:25.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/99845980' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-16T08:45:25.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:25 smithi160 ceph-mon[30137]: osdmap e52: 8 total, 8 up, 8 in 2024-09-16T08:45:25.723 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":51,"num_osds":8,"num_up_osds":8,"osd_up_since":1726476323,"num_in_osds":8,"osd_in_since":1726476300,"num_remapped_pgs":0} 2024-09-16T08:45:25.723 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T08:45:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:25 smithi169 ceph-mon[33465]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:25 smithi169 ceph-mon[33465]: Detected new or changed devices on smithi169 2024-09-16T08:45:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:25 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:25 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:25 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:45:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:25 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T08:45:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:25 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T08:45:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:25 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T08:45:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:25 smithi169 ceph-mon[33465]: Adjusting osd_memory_target on smithi169 to 4272M 2024-09-16T08:45:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:25 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:25.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:25 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:25.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:25 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:45:25.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:25 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:25.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/99845980' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-16T08:45:25.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:25 smithi169 ceph-mon[33465]: osdmap e52: 8 total, 8 up, 8 in 2024-09-16T08:45:25.966 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:26.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.402+0000 7faf96df9640 1 -- 172.21.15.160:0/223758826 >> v1:172.21.15.160:6790/0 conn(0x7faf9010b860 legacy=0x7faf9010dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:26.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.402+0000 7faf96df9640 1 -- 172.21.15.160:0/223758826 shutdown_connections 2024-09-16T08:45:26.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.402+0000 7faf96df9640 1 -- 172.21.15.160:0/223758826 >> 172.21.15.160:0/223758826 conn(0x7faf900fd450 msgr2=0x7faf900ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:26.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.402+0000 7faf96df9640 1 -- 172.21.15.160:0/223758826 shutdown_connections 2024-09-16T08:45:26.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.402+0000 7faf96df9640 1 -- 172.21.15.160:0/223758826 wait complete. 2024-09-16T08:45:26.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.402+0000 7faf96df9640 1 Processor -- start 2024-09-16T08:45:26.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.402+0000 7faf96df9640 1 -- start start 2024-09-16T08:45:26.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf96df9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7faf901a7800 con 0x7faf9010f370 2024-09-16T08:45:26.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf96df9640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7faf901a8a00 con 0x7faf90103880 2024-09-16T08:45:26.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf96df9640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7faf901a9c00 con 0x7faf9010b860 2024-09-16T08:45:26.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8ffff640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7faf9010b860 0x7faf901a2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:49618/0 (socket says 172.21.15.160:49618) 2024-09-16T08:45:26.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8ffff640 1 -- 172.21.15.160:0/124783958 learned_addr learned my addr 172.21.15.160:0/124783958 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:45:26.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf95396640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7faf9010f370 0x7faf901a5f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:37338/0 (socket says 172.21.15.160:37338) 2024-09-16T08:45:26.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1294192114 0 0) 0x7faf901a9c00 con 0x7faf9010b860 2024-09-16T08:45:26.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf80003520 con 0x7faf9010b860 2024-09-16T08:45:26.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 975249139 0 0) 0x7faf901a7800 con 0x7faf9010f370 2024-09-16T08:45:26.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf901a9c00 con 0x7faf9010f370 2024-09-16T08:45:26.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3275763162 0 0) 0x7faf901a8a00 con 0x7faf90103880 2024-09-16T08:45:26.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf901a7800 con 0x7faf90103880 2024-09-16T08:45:26.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1144259231 0 0) 0x7faf80003520 con 0x7faf9010b860 2024-09-16T08:45:26.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7faf901a8a00 con 0x7faf9010b860 2024-09-16T08:45:26.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7faf7c0015d0 con 0x7faf9010b860 2024-09-16T08:45:26.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1939738188 0 0) 0x7faf901a9c00 con 0x7faf9010f370 2024-09-16T08:45:26.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7faf80003520 con 0x7faf9010f370 2024-09-16T08:45:26.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3616608244 0 0) 0x7faf901a7800 con 0x7faf90103880 2024-09-16T08:45:26.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7faf901a9c00 con 0x7faf90103880 2024-09-16T08:45:26.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7faf88001590 con 0x7faf9010f370 2024-09-16T08:45:26.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7faf84001830 con 0x7faf90103880 2024-09-16T08:45:26.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.403+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1075800777 0 0) 0x7faf80003520 con 0x7faf9010f370 2024-09-16T08:45:26.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.404+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 >> v1:172.21.15.160:6790/0 conn(0x7faf9010b860 legacy=0x7faf901a2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:26.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.404+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 >> v1:172.21.15.169:6789/0 conn(0x7faf90103880 legacy=0x7faf9010afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:26.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.404+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf901aaea0 con 0x7faf9010f370 2024-09-16T08:45:26.408 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.404+0000 7faf96df9640 1 -- 172.21.15.160:0/124783958 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7faf901a7a30 con 0x7faf9010f370 2024-09-16T08:45:26.408 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.404+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7faf88003de0 con 0x7faf9010f370 2024-09-16T08:45:26.408 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.404+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7faf88005d90 con 0x7faf9010f370 2024-09-16T08:45:26.408 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.404+0000 7faf96df9640 1 -- 172.21.15.160:0/124783958 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7faf901a8060 con 0x7faf9010f370 2024-09-16T08:45:26.408 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.405+0000 7faf96df9640 1 -- 172.21.15.160:0/124783958 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf9006b260 con 0x7faf9010f370 2024-09-16T08:45:26.408 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.405+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 15) ==== 103045+0+0 (unknown 4003825774 0 0) 0x7faf88003970 con 0x7faf9010f370 2024-09-16T08:45:26.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.406+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(52..52 src has 1..52) ==== 4081+0+0 (unknown 908256175 0 0) 0x7faf88098ba0 con 0x7faf9010f370 2024-09-16T08:45:26.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.410+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7faf880605f0 con 0x7faf9010f370 2024-09-16T08:45:26.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.586+0000 7faf96df9640 1 -- 172.21.15.160:0/124783958 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf90115020 con 0x7faf9010f370 2024-09-16T08:45:26.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.586+0000 7faf8dffb640 1 -- 172.21.15.160:0/124783958 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+11557 (unknown 2143218748 0 150475440) 0x7faf880649d0 con 0x7faf9010f370 2024-09-16T08:45:26.588 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:45:26.588 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":52,"fsid":"27cca2fc-7407-11ef-bceb-c7b262605968","created":"2024-09-16T08:39:54.060780+0000","modified":"2024-09-16T08:45:25.065607+0000","last_up_change":"2024-09-16T08:45:23.119335+0000","last_in_change":"2024-09-16T08:45:00.473379+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T08:43:02.103619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e77e8c1-25c0-4842-90a7-48769d62dbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6801","nonce":520428698}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6802","nonce":520428698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6804","nonce":520428698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6803","nonce":520428698}]},"public_addr":"172.21.15.160:6801/520428698","cluster_addr":"172.21.15.160:6802/520428698","heartbeat_back_addr":"172.21.15.160:6804/520428698","heartbeat_front_addr":"172.21.15.160:6803/520428698","state":["exists","up"]},{"osd":1,"uuid":"08f4dc4d-2873-45f3-a1d9-7c12f6e2c995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6805","nonce":704046158}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6806","nonce":704046158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6808","nonce":704046158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6807","nonce":704046158}]},"public_addr":"172.21.15.160:6805/704046158","cluster_addr":"172.21.15.160:6806/704046158","heartbeat_back_addr":"172.21.15.160:6808/704046158","heartbeat_front_addr":"172.21.15.160:6807/704046158","state":["exists","up"]},{"osd":2,"uuid":"0ba6c8a1-5676-4e1d-994a-a44a9b7d1996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6809","nonce":2138225188}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6810","nonce":2138225188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6812","nonce":2138225188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6811","nonce":2138225188}]},"public_addr":"172.21.15.160:6809/2138225188","cluster_addr":"172.21.15.160:6810/2138225188","heartbeat_back_addr":"172.21.15.160:6812/2138225188","heartbeat_front_addr":"172.21.15.160:6811/2138225188","state":["exists","up"]},{"osd":3,"uuid":"01400bbe-6dfe-4789-8b07-5bb3177eddfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6813","nonce":1752204566}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6814","nonce":1752204566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6816","nonce":1752204566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6815","nonce":1752204566}]},"public_addr":"172.21.15.160:6813/1752204566","cluster_addr":"172.21.15.160:6814/1752204566","heartbeat_back_addr":"172.21.15.160:6816/1752204566","heartbeat_front_addr":"172.21.15.160:6815/1752204566","state":["exists","up"]},{"osd":4,"uuid":"f0b51ca8-0851-4938-a811-667d57dbc18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6800","nonce":1893879009}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6801","nonce":1893879009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6803","nonce":1893879009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6802","nonce":1893879009}]},"public_addr":"172.21.15.169:6800/1893879009","cluster_addr":"172.21.15.169:6801/1893879009","heartbeat_back_addr":"172.21.15.169:6803/1893879009","heartbeat_front_addr":"172.21.15.169:6802/1893879009","state":["exists","up"]},{"osd":5,"uuid":"15a1f6ce-7c1b-4ebd-ab8a-4bacdbe5df55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6804","nonce":4217518758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6805","nonce":4217518758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6807","nonce":4217518758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6806","nonce":4217518758}]},"public_addr":"172.21.15.169:6804/4217518758","cluster_addr":"172.21.15.169:6805/4217518758","heartbeat_back_addr":"172.21.15.169:6807/4217518758","heartbeat_front_addr":"172.21.15.169:6806/4217518758","state":["exists","up"]},{"osd":6,"uuid":"c3a03867-0d98-41bb-bbd7-cd6fb4f67eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6808","nonce":3462109725}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6809","nonce":3462109725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6811","nonce":3462109725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6810","nonce":3462109725}]},"public_addr":"172.21.15.169:6808/3462109725","cluster_addr":"172.21.15.169:6809/3462109725","heartbeat_back_addr":"172.21.15.169:6811/3462109725","heartbeat_front_addr":"172.21.15.169:6810/3462109725","state":["exists","up"]},{"osd":7,"uuid":"a24f20d7-85f2-4c15-bab9-05382f876001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6812","nonce":1196718305}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6813","nonce":1196718305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6815","nonce":1196718305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6814","nonce":1196718305}]},"public_addr":"172.21.15.169:6812/1196718305","cluster_addr":"172.21.15.169:6813/1196718305","heartbeat_back_addr":"172.21.15.169:6815/1196718305","heartbeat_front_addr":"172.21.15.169:6814/1196718305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:42:02.669441+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:42:30.592545+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:42:58.969509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:43:26.692920+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:43:53.561518+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:44:22.355115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:44:51.383158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:45:21.104100+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/797617578":"2024-09-17T08:40:45.804394+0000","172.21.15.160:0/2198066550":"2024-09-17T08:40:45.804394+0000","172.21.15.160:0/3719822124":"2024-09-17T08:40:45.804394+0000","172.21.15.160:6800/3425583652":"2024-09-17T08:40:45.804394+0000","172.21.15.160:0/979267250":"2024-09-17T08:40:20.860608+0000","172.21.15.160:0/17466509":"2024-09-17T08:40:20.860608+0000","172.21.15.160:6800/2809527782":"2024-09-17T08:40:20.860608+0000","172.21.15.160:0/412999819":"2024-09-17T08:40:20.860608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T08:45:26.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.589+0000 7faf96df9640 1 -- 172.21.15.160:0/124783958 >> v1:172.21.15.160:6800/1286008384 conn(0x7faf8007bb70 legacy=0x7faf8007e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:26.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.589+0000 7faf96df9640 1 -- 172.21.15.160:0/124783958 >> v1:172.21.15.160:6789/0 conn(0x7faf9010f370 legacy=0x7faf901a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:26.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.589+0000 7faf96df9640 1 -- 172.21.15.160:0/124783958 shutdown_connections 2024-09-16T08:45:26.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.589+0000 7faf96df9640 1 -- 172.21.15.160:0/124783958 >> 172.21.15.160:0/124783958 conn(0x7faf900fd450 msgr2=0x7faf900ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:26.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.589+0000 7faf96df9640 1 -- 172.21.15.160:0/124783958 shutdown_connections 2024-09-16T08:45:26.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:26.589+0000 7faf96df9640 1 -- 172.21.15.160:0/124783958 wait complete. 2024-09-16T08:45:27.305 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T08:43:02.103619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T08:45:27.306 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-16T08:45:27.553 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:27.670 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:27 smithi160 ceph-mon[30137]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:27.670 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/124783958' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T08:45:27.670 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:27 smithi160 ceph-mon[40028]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:27.670 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/124783958' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T08:45:27.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:27 smithi169 ceph-mon[33465]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:27.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/124783958' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T08:45:27.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.962+0000 7f97e5e1d640 1 -- 172.21.15.160:0/4032708049 >> v1:172.21.15.160:6789/0 conn(0x7f97e010a570 legacy=0x7f97e010c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:27.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.962+0000 7f97e5e1d640 1 -- 172.21.15.160:0/4032708049 shutdown_connections 2024-09-16T08:45:27.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.962+0000 7f97e5e1d640 1 -- 172.21.15.160:0/4032708049 >> 172.21.15.160:0/4032708049 conn(0x7f97e00fd450 msgr2=0x7f97e00ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:27.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.962+0000 7f97e5e1d640 1 -- 172.21.15.160:0/4032708049 shutdown_connections 2024-09-16T08:45:27.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.962+0000 7f97e5e1d640 1 -- 172.21.15.160:0/4032708049 wait complete. 2024-09-16T08:45:27.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.963+0000 7f97e5e1d640 1 Processor -- start 2024-09-16T08:45:27.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.963+0000 7f97e5e1d640 1 -- start start 2024-09-16T08:45:27.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.963+0000 7f97e5e1d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f97e01a9910 con 0x7f97e010a570 2024-09-16T08:45:27.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.963+0000 7f97e5e1d640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f97e01aab10 con 0x7f97e0101a00 2024-09-16T08:45:27.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.963+0000 7f97e5e1d640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f97e01abd10 con 0x7f97e010e110 2024-09-16T08:45:27.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.963+0000 7f97dffff640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f97e010e110 0x7f97e01a8090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:47462/0 (socket says 172.21.15.160:47462) 2024-09-16T08:45:27.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.963+0000 7f97deffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f97e010a570 0x7f97e01a4960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:39524/0 (socket says 172.21.15.160:39524) 2024-09-16T08:45:27.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.963+0000 7f97dffff640 1 -- 172.21.15.160:0/1370752268 learned_addr learned my addr 172.21.15.160:0/1370752268 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:45:27.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.963+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2155755658 0 0) 0x7f97e01abd10 con 0x7f97e010e110 2024-09-16T08:45:27.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.963+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97cc003520 con 0x7f97e010e110 2024-09-16T08:45:27.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.963+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3181031453 0 0) 0x7f97e01a9910 con 0x7f97e010a570 2024-09-16T08:45:27.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.963+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97e01abd10 con 0x7f97e010a570 2024-09-16T08:45:27.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.963+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4021749081 0 0) 0x7f97e01aab10 con 0x7f97e0101a00 2024-09-16T08:45:27.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.963+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97e01a9910 con 0x7f97e0101a00 2024-09-16T08:45:27.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3790836451 0 0) 0x7f97cc003520 con 0x7f97e010e110 2024-09-16T08:45:27.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f97e01aab10 con 0x7f97e010e110 2024-09-16T08:45:27.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f97d8001d10 con 0x7f97e010e110 2024-09-16T08:45:27.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 686322594 0 0) 0x7f97e01abd10 con 0x7f97e010a570 2024-09-16T08:45:27.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f97cc003520 con 0x7f97e010a570 2024-09-16T08:45:27.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f97c80013e0 con 0x7f97e010a570 2024-09-16T08:45:27.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3708343857 0 0) 0x7f97e01a9910 con 0x7f97e0101a00 2024-09-16T08:45:27.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f97e01abd10 con 0x7f97e0101a00 2024-09-16T08:45:27.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f97d0001830 con 0x7f97e0101a00 2024-09-16T08:45:27.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1096952859 0 0) 0x7f97e01aab10 con 0x7f97e010e110 2024-09-16T08:45:27.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 >> v1:172.21.15.169:6789/0 conn(0x7f97e0101a00 legacy=0x7f97e010ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:27.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 >> v1:172.21.15.160:6789/0 conn(0x7f97e010a570 legacy=0x7f97e01a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:27.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e01acfb0 con 0x7f97e010e110 2024-09-16T08:45:27.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97e5e1d640 1 -- 172.21.15.160:0/1370752268 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f97e01abf40 con 0x7f97e010e110 2024-09-16T08:45:27.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97e5e1d640 1 -- 172.21.15.160:0/1370752268 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f97e01ac570 con 0x7f97e010e110 2024-09-16T08:45:27.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f97d8003e00 con 0x7f97e010e110 2024-09-16T08:45:28.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.964+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f97d8005fd0 con 0x7f97e010e110 2024-09-16T08:45:28.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.965+0000 7f97e5e1d640 1 -- 172.21.15.160:0/1370752268 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c00050b0 con 0x7f97e010e110 2024-09-16T08:45:28.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.966+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 15) ==== 103045+0+0 (unknown 4003825774 0 0) 0x7f97d8003990 con 0x7f97e010e110 2024-09-16T08:45:28.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.966+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(52..52 src has 1..52) ==== 4081+0+0 (unknown 908256175 0 0) 0x7f97d8098b20 con 0x7f97e010e110 2024-09-16T08:45:28.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:27.970+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f97d80605b0 con 0x7f97e010e110 2024-09-16T08:45:28.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:28.144+0000 7f97e5e1d640 1 -- 172.21.15.160:0/1370752268 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f97c0005c70 con 0x7f97e010e110 2024-09-16T08:45:28.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:28.145+0000 7f97dcff9640 1 -- 172.21.15.160:0/1370752268 <== mon.2 v1:172.21.15.160:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (unknown 2313579657 0 2170607528) 0x7f97d8064990 con 0x7f97e010e110 2024-09-16T08:45:28.146 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-09-16T08:45:28.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:28.147+0000 7f97e5e1d640 1 -- 172.21.15.160:0/1370752268 >> v1:172.21.15.160:6800/1286008384 conn(0x7f97cc07bbd0 legacy=0x7f97cc07e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:28.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:28.147+0000 7f97e5e1d640 1 -- 172.21.15.160:0/1370752268 >> v1:172.21.15.160:6790/0 conn(0x7f97e010e110 legacy=0x7f97e01a8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:28.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:28.147+0000 7f97e5e1d640 1 -- 172.21.15.160:0/1370752268 shutdown_connections 2024-09-16T08:45:28.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:28.147+0000 7f97e5e1d640 1 -- 172.21.15.160:0/1370752268 >> 172.21.15.160:0/1370752268 conn(0x7f97e00fd450 msgr2=0x7f97e01049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:28.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:28.147+0000 7f97e5e1d640 1 -- 172.21.15.160:0/1370752268 shutdown_connections 2024-09-16T08:45:28.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:28.147+0000 7f97e5e1d640 1 -- 172.21.15.160:0/1370752268 wait complete. 2024-09-16T08:45:28.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1370752268' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-16T08:45:28.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1370752268' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-16T08:45:28.837 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi160 2024-09-16T08:45:28.837 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi160=foo.a' 2024-09-16T08:45:28.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1370752268' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-16T08:45:29.098 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:45:29.485 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:29 smithi169 ceph-mon[33465]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:29.585 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.570+0000 7fa7696ef640 1 -- 172.21.15.169:0/959265577 >> v1:172.21.15.169:6789/0 conn(0x7fa76406b7d0 legacy=0x7fa764105380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:29.585 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.570+0000 7fa7696ef640 1 -- 172.21.15.169:0/959265577 shutdown_connections 2024-09-16T08:45:29.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.570+0000 7fa7696ef640 1 -- 172.21.15.169:0/959265577 >> 172.21.15.169:0/959265577 conn(0x7fa7640fcf70 msgr2=0x7fa7640ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:29.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.570+0000 7fa7696ef640 1 -- 172.21.15.169:0/959265577 shutdown_connections 2024-09-16T08:45:29.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.570+0000 7fa7696ef640 1 -- 172.21.15.169:0/959265577 wait complete. 2024-09-16T08:45:29.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.571+0000 7fa7696ef640 1 Processor -- start 2024-09-16T08:45:29.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.571+0000 7fa7696ef640 1 -- start start 2024-09-16T08:45:29.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.571+0000 7fa7696ef640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa7641a2ec0 con 0x7fa76410a8b0 2024-09-16T08:45:29.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.571+0000 7fa7696ef640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa7641a40a0 con 0x7fa76406b7d0 2024-09-16T08:45:29.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.571+0000 7fa7696ef640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa7641a5280 con 0x7fa764106b30 2024-09-16T08:45:29.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.571+0000 7fa762ffd640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fa76406b7d0 0x7fa764197e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:49424/0 (socket says 172.21.15.169:49424) 2024-09-16T08:45:29.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.571+0000 7fa762ffd640 1 -- 172.21.15.169:0/2293578930 learned_addr learned my addr 172.21.15.169:0/2293578930 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:45:29.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.571+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3861456898 0 0) 0x7fa7641a40a0 con 0x7fa76406b7d0 2024-09-16T08:45:29.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.571+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa750003520 con 0x7fa76406b7d0 2024-09-16T08:45:29.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.571+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1877591590 0 0) 0x7fa7641a5280 con 0x7fa764106b30 2024-09-16T08:45:29.587 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.571+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7641a40a0 con 0x7fa764106b30 2024-09-16T08:45:29.587 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.571+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2884989068 0 0) 0x7fa7641a2ec0 con 0x7fa76410a8b0 2024-09-16T08:45:29.587 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.571+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7641a5280 con 0x7fa76410a8b0 2024-09-16T08:45:29.587 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.572+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3563661297 0 0) 0x7fa750003520 con 0x7fa76406b7d0 2024-09-16T08:45:29.587 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.572+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa7641a2ec0 con 0x7fa76406b7d0 2024-09-16T08:45:29.587 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.572+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fa754001e70 con 0x7fa76406b7d0 2024-09-16T08:45:29.587 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.572+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 <== mon.1 v1:172.21.15.169:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3324057443 0 0) 0x7fa7641a2ec0 con 0x7fa76406b7d0 2024-09-16T08:45:29.587 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.572+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 >> v1:172.21.15.160:6790/0 conn(0x7fa764106b30 legacy=0x7fa76419e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:29.587 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.572+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 >> v1:172.21.15.160:6789/0 conn(0x7fa76410a8b0 legacy=0x7fa7641a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:29.587 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.572+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7641a6460 con 0x7fa76406b7d0 2024-09-16T08:45:29.587 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.572+0000 7fa7696ef640 1 -- 172.21.15.169:0/2293578930 --> v1:172.21.15.169:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa7641a5450 con 0x7fa76406b7d0 2024-09-16T08:45:29.587 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.572+0000 7fa7696ef640 1 -- 172.21.15.169:0/2293578930 --> v1:172.21.15.169:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa7641a5a80 con 0x7fa76406b7d0 2024-09-16T08:45:29.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.572+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 <== mon.1 v1:172.21.15.169:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fa7540020a0 con 0x7fa76406b7d0 2024-09-16T08:45:29.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.572+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 <== mon.1 v1:172.21.15.169:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fa754005120 con 0x7fa76406b7d0 2024-09-16T08:45:29.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.573+0000 7fa7696ef640 1 -- 172.21.15.169:0/2293578930 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7440050b0 con 0x7fa76406b7d0 2024-09-16T08:45:29.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.573+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 <== mon.1 v1:172.21.15.169:6789/0 7 ==== mgrmap(e 15) ==== 103045+0+0 (unknown 4003825774 0 0) 0x7fa754003b80 con 0x7fa76406b7d0 2024-09-16T08:45:29.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.574+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 <== mon.1 v1:172.21.15.169:6789/0 8 ==== osd_map(52..52 src has 1..52) ==== 4081+0+0 (unknown 908256175 0 0) 0x7fa754098e00 con 0x7fa76406b7d0 2024-09-16T08:45:29.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.578+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 <== mon.1 v1:172.21.15.169:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fa754060990 con 0x7fa76406b7d0 2024-09-16T08:45:29.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:29 smithi160 ceph-mon[30137]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:29.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:29 smithi160 ceph-mon[40028]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:29.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.763+0000 7fa7696ef640 1 -- 172.21.15.169:0/2293578930 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi160=foo.a", "target": ["mon-mgr", ""]}) -- 0x7fa744000bc0 con 0x7fa75007b4b0 2024-09-16T08:45:29.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.771+0000 7fa75bfff640 1 -- 172.21.15.169:0/2293578930 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (unknown 0 0 1123153589) 0x7fa744000bc0 con 0x7fa75007b4b0 2024-09-16T08:45:29.772 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled rgw.foo.a update... 2024-09-16T08:45:29.773 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.773+0000 7fa7696ef640 1 -- 172.21.15.169:0/2293578930 >> v1:172.21.15.160:6800/1286008384 conn(0x7fa75007b4b0 legacy=0x7fa75007d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:29.774 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.773+0000 7fa7696ef640 1 -- 172.21.15.169:0/2293578930 >> v1:172.21.15.169:6789/0 conn(0x7fa76406b7d0 legacy=0x7fa764197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:29.774 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.773+0000 7fa7696ef640 1 -- 172.21.15.169:0/2293578930 shutdown_connections 2024-09-16T08:45:29.774 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.773+0000 7fa7696ef640 1 -- 172.21.15.169:0/2293578930 >> 172.21.15.169:0/2293578930 conn(0x7fa7640fcf70 msgr2=0x7fa764108fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:29.774 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.773+0000 7fa7696ef640 1 -- 172.21.15.169:0/2293578930 shutdown_connections 2024-09-16T08:45:29.774 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:29.773+0000 7fa7696ef640 1 -- 172.21.15.169:0/2293578930 wait complete. 2024-09-16T08:45:30.393 DEBUG:teuthology.orchestra.run.smithi160:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@rgw.foo.a.service 2024-09-16T08:45:30.396 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi169 2024-09-16T08:45:30.396 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-16T08:45:30.681 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:45:31.032 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[40028]: from='client.24364 v1:172.21.15.169:0/2293578930' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi160=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:31.032 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[40028]: Saving service rgw.foo.a spec with placement smithi160=foo.a;count:1 2024-09-16T08:45:31.032 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:31.032 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:45:31.033 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:31.033 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:45:31.033 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:31.033 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-16T08:45:31.033 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T08:45:31.033 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:31.033 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:31.033 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[30137]: from='client.24364 v1:172.21.15.169:0/2293578930' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi160=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:31.033 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[30137]: Saving service rgw.foo.a spec with placement smithi160=foo.a;count:1 2024-09-16T08:45:31.033 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:31.033 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:45:31.034 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:31.034 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:45:31.034 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:31.034 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-16T08:45:31.034 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T08:45:31.034 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:31.034 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:30 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:31.060 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:30 smithi169 ceph-mon[33465]: from='client.24364 v1:172.21.15.169:0/2293578930' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi160=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:31.060 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:30 smithi169 ceph-mon[33465]: Saving service rgw.foo.a spec with placement smithi160=foo.a;count:1 2024-09-16T08:45:31.060 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:31.060 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:45:31.060 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:31.061 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:45:31.061 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:31.061 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-16T08:45:31.061 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T08:45:31.061 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:31.061 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:30 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:31.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.143+0000 7fd56efb4640 1 -- 172.21.15.169:0/1562181426 >> v1:172.21.15.169:6789/0 conn(0x7fd568106c30 legacy=0x7fd568109020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:31.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.143+0000 7fd56efb4640 1 -- 172.21.15.169:0/1562181426 shutdown_connections 2024-09-16T08:45:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.143+0000 7fd56efb4640 1 -- 172.21.15.169:0/1562181426 >> 172.21.15.169:0/1562181426 conn(0x7fd5680fe0a0 msgr2=0x7fd5681004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.144+0000 7fd56efb4640 1 -- 172.21.15.169:0/1562181426 shutdown_connections 2024-09-16T08:45:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.144+0000 7fd56efb4640 1 -- 172.21.15.169:0/1562181426 wait complete. 2024-09-16T08:45:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.144+0000 7fd56efb4640 1 Processor -- start 2024-09-16T08:45:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.144+0000 7fd56efb4640 1 -- start start 2024-09-16T08:45:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.144+0000 7fd56efb4640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd5681a4f80 con 0x7fd5681027b0 2024-09-16T08:45:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.144+0000 7fd56efb4640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd5681a6180 con 0x7fd56810a8c0 2024-09-16T08:45:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.144+0000 7fd56efb4640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd5681a7380 con 0x7fd568106c30 2024-09-16T08:45:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.144+0000 7fd56d551640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fd56810a8c0 0x7fd5681a3680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:49446/0 (socket says 172.21.15.169:49446) 2024-09-16T08:45:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.144+0000 7fd56d551640 1 -- 172.21.15.169:0/3072166922 learned_addr learned my addr 172.21.15.169:0/3072166922 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:45:31.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4108351535 0 0) 0x7fd5681a6180 con 0x7fd56810a8c0 2024-09-16T08:45:31.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd558003520 con 0x7fd56810a8c0 2024-09-16T08:45:31.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3767265487 0 0) 0x7fd5681a4f80 con 0x7fd5681027b0 2024-09-16T08:45:31.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5681a6180 con 0x7fd5681027b0 2024-09-16T08:45:31.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2686953011 0 0) 0x7fd5681a7380 con 0x7fd568106c30 2024-09-16T08:45:31.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5681a4f80 con 0x7fd568106c30 2024-09-16T08:45:31.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2836584237 0 0) 0x7fd558003520 con 0x7fd56810a8c0 2024-09-16T08:45:31.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd5681a7380 con 0x7fd56810a8c0 2024-09-16T08:45:31.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fd560001cb0 con 0x7fd56810a8c0 2024-09-16T08:45:31.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3929728923 0 0) 0x7fd5681a6180 con 0x7fd5681027b0 2024-09-16T08:45:31.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd558003520 con 0x7fd5681027b0 2024-09-16T08:45:31.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fd55c0018b0 con 0x7fd5681027b0 2024-09-16T08:45:31.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 <== mon.1 v1:172.21.15.169:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3638738835 0 0) 0x7fd5681a7380 con 0x7fd56810a8c0 2024-09-16T08:45:31.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 >> v1:172.21.15.160:6790/0 conn(0x7fd568106c30 legacy=0x7fd56819fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:31.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 >> v1:172.21.15.160:6789/0 conn(0x7fd5681027b0 legacy=0x7fd568076bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:31.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5681a8620 con 0x7fd56810a8c0 2024-09-16T08:45:31.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd56efb4640 1 -- 172.21.15.169:0/3072166922 --> v1:172.21.15.169:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd5681a51b0 con 0x7fd56810a8c0 2024-09-16T08:45:31.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 <== mon.1 v1:172.21.15.169:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fd560003e00 con 0x7fd56810a8c0 2024-09-16T08:45:31.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 <== mon.1 v1:172.21.15.169:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fd560005f60 con 0x7fd56810a8c0 2024-09-16T08:45:31.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.145+0000 7fd56efb4640 1 -- 172.21.15.169:0/3072166922 --> v1:172.21.15.169:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd5681a57e0 con 0x7fd56810a8c0 2024-09-16T08:45:31.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.146+0000 7fd56efb4640 1 -- 172.21.15.169:0/3072166922 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd56810fa50 con 0x7fd56810a8c0 2024-09-16T08:45:31.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.147+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 <== mon.1 v1:172.21.15.169:6789/0 7 ==== mgrmap(e 15) ==== 103045+0+0 (unknown 4003825774 0 0) 0x7fd560003990 con 0x7fd56810a8c0 2024-09-16T08:45:31.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.147+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 <== mon.1 v1:172.21.15.169:6789/0 8 ==== osd_map(52..52 src has 1..52) ==== 4081+0+0 (unknown 908256175 0 0) 0x7fd560098c50 con 0x7fd56810a8c0 2024-09-16T08:45:31.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.151+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 <== mon.1 v1:172.21.15.169:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fd5600606a0 con 0x7fd56810a8c0 2024-09-16T08:45:31.327 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.327+0000 7fd56efb4640 1 -- 172.21.15.169:0/3072166922 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) -- 0x7fd56810fc20 con 0x7fd56810a8c0 2024-09-16T08:45:31.610 INFO:journalctl@ceph.rgw.foo.a.smithi160.stdout:Sep 16 08:45:31 smithi160 systemd[1]: Starting Ceph rgw.foo.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T08:45:31.796 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.795+0000 7fd565ffb640 1 -- 172.21.15.169:0/3072166922 <== mon.1 v1:172.21.15.169:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v53) ==== 160+0+0 (unknown 3942856737 0 0) 0x7fd560064a80 con 0x7fd56810a8c0 2024-09-16T08:45:31.796 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'datapool' created 2024-09-16T08:45:31.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.798+0000 7fd56efb4640 1 -- 172.21.15.169:0/3072166922 >> v1:172.21.15.160:6800/1286008384 conn(0x7fd55807b810 legacy=0x7fd55807dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:31.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.798+0000 7fd56efb4640 1 -- 172.21.15.169:0/3072166922 >> v1:172.21.15.169:6789/0 conn(0x7fd56810a8c0 legacy=0x7fd5681a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:31.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.798+0000 7fd56efb4640 1 -- 172.21.15.169:0/3072166922 shutdown_connections 2024-09-16T08:45:31.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.798+0000 7fd56efb4640 1 -- 172.21.15.169:0/3072166922 >> 172.21.15.169:0/3072166922 conn(0x7fd5680fe0a0 msgr2=0x7fd5680fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:31.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.798+0000 7fd56efb4640 1 -- 172.21.15.169:0/3072166922 shutdown_connections 2024-09-16T08:45:31.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:31.798+0000 7fd56efb4640 1 -- 172.21.15.169:0/3072166922 wait complete. 2024-09-16T08:45:31.867 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:31 smithi160 ceph-mon[30137]: Deploying daemon rgw.foo.a on smithi160 2024-09-16T08:45:31.867 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:31 smithi160 ceph-mon[30137]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:31.868 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.169:0/3072166922' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-16T08:45:31.868 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:31 smithi160 ceph-mon[30137]: from='client.24370 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-16T08:45:31.868 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:31 smithi160 ceph-mon[40028]: Deploying daemon rgw.foo.a on smithi160 2024-09-16T08:45:31.868 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:31 smithi160 ceph-mon[40028]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:31.868 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.169:0/3072166922' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-16T08:45:31.868 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:31 smithi160 ceph-mon[40028]: from='client.24370 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-16T08:45:31.869 INFO:journalctl@ceph.rgw.foo.a.smithi160.stdout:Sep 16 08:45:31 smithi160 podman[66226]: 2024-09-16 08:45:31.742719918 +0000 UTC m=+0.111492005 container create 2591e5b22df76127c612bbd6641334c0c32f421dafdd3282c4cfa2d6c80d2c59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-rgw-foo-a, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T08:45:31.869 INFO:journalctl@ceph.rgw.foo.a.smithi160.stdout:Sep 16 08:45:31 smithi160 podman[66226]: 2024-09-16 08:45:31.647065333 +0000 UTC m=+0.015837415 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T08:45:32.081 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:31 smithi169 ceph-mon[33465]: Deploying daemon rgw.foo.a on smithi160 2024-09-16T08:45:32.081 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:31 smithi169 ceph-mon[33465]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:32.081 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.169:0/3072166922' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-16T08:45:32.081 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:31 smithi169 ceph-mon[33465]: from='client.24370 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-16T08:45:32.418 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- rbd pool init datapool 2024-09-16T08:45:32.447 INFO:journalctl@ceph.rgw.foo.a.smithi160.stdout:Sep 16 08:45:31 smithi160 podman[66226]: 2024-09-16 08:45:31.995679219 +0000 UTC m=+0.364451303 container init 2591e5b22df76127c612bbd6641334c0c32f421dafdd3282c4cfa2d6c80d2c59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-16T08:45:32.447 INFO:journalctl@ceph.rgw.foo.a.smithi160.stdout:Sep 16 08:45:32 smithi160 podman[66226]: 2024-09-16 08:45:32.001156598 +0000 UTC m=+0.369928694 container start 2591e5b22df76127c612bbd6641334c0c32f421dafdd3282c4cfa2d6c80d2c59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T08:45:32.448 INFO:journalctl@ceph.rgw.foo.a.smithi160.stdout:Sep 16 08:45:32 smithi160 bash[66226]: 2591e5b22df76127c612bbd6641334c0c32f421dafdd3282c4cfa2d6c80d2c59 2024-09-16T08:45:32.448 INFO:journalctl@ceph.rgw.foo.a.smithi160.stdout:Sep 16 08:45:32 smithi160 systemd[1]: Started Ceph rgw.foo.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:45:32.668 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:45:33.049 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[30137]: from='client.24370 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-16T08:45:33.049 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[30137]: osdmap e53: 8 total, 8 up, 8 in 2024-09-16T08:45:33.050 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[30137]: pgmap v186: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:33.050 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:33.050 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:33.050 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:33.050 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[30137]: Saving service rgw.foo.a spec with placement smithi160=foo.a;count:1 2024-09-16T08:45:33.050 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:33.050 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:33.050 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:45:33.050 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[40028]: from='client.24370 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-16T08:45:33.050 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[40028]: osdmap e53: 8 total, 8 up, 8 in 2024-09-16T08:45:33.050 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[40028]: pgmap v186: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:33.051 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:33.051 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:33.051 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:33.051 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[40028]: Saving service rgw.foo.a spec with placement smithi160=foo.a;count:1 2024-09-16T08:45:33.051 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:33.051 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:33.051 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:32 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:45:33.082 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:32 smithi169 ceph-mon[33465]: from='client.24370 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-16T08:45:33.083 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:32 smithi169 ceph-mon[33465]: osdmap e53: 8 total, 8 up, 8 in 2024-09-16T08:45:33.083 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:32 smithi169 ceph-mon[33465]: pgmap v186: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:33.083 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:33.083 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:33.083 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:33.083 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:32 smithi169 ceph-mon[33465]: Saving service rgw.foo.a spec with placement smithi160=foo.a;count:1 2024-09-16T08:45:33.083 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:33.083 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:33.083 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:32 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:45:34.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:33 smithi169 ceph-mon[33465]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:45:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:33 smithi169 ceph-mon[33465]: osdmap e54: 8 total, 8 up, 8 in 2024-09-16T08:45:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3135351515' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-16T08:45:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:33 smithi169 ceph-mon[33465]: from='client.34373 ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-16T08:45:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.169:0/362065114' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-16T08:45:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:33 smithi169 ceph-mon[33465]: from='client.24385 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-16T08:45:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:33 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:45:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:33 smithi169 ceph-mon[33465]: Checking dashboard <-> RGW credentials 2024-09-16T08:45:34.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[40028]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:45:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[40028]: osdmap e54: 8 total, 8 up, 8 in 2024-09-16T08:45:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3135351515' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-16T08:45:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[40028]: from='client.34373 ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-16T08:45:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.169:0/362065114' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-16T08:45:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[40028]: from='client.24385 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-16T08:45:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:45:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[40028]: Checking dashboard <-> RGW credentials 2024-09-16T08:45:34.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[30137]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:45:34.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[30137]: osdmap e54: 8 total, 8 up, 8 in 2024-09-16T08:45:34.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3135351515' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-16T08:45:34.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[30137]: from='client.34373 ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-16T08:45:34.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.169:0/362065114' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-16T08:45:34.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[30137]: from='client.24385 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-16T08:45:34.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:34.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:34.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:34.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:45:34.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:33 smithi160 ceph-mon[30137]: Checking dashboard <-> RGW credentials 2024-09-16T08:45:35.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:34 smithi169 ceph-mon[33465]: from='client.34373 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T08:45:35.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:34 smithi169 ceph-mon[33465]: from='client.24385 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-16T08:45:35.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:34 smithi169 ceph-mon[33465]: osdmap e55: 8 total, 8 up, 8 in 2024-09-16T08:45:35.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:34 smithi169 ceph-mon[33465]: pgmap v189: 36 pgs: 11 creating+peering, 2 active+clean, 23 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:35.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:34 smithi160 ceph-mon[30137]: from='client.34373 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T08:45:35.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:34 smithi160 ceph-mon[30137]: from='client.24385 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-16T08:45:35.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:34 smithi160 ceph-mon[30137]: osdmap e55: 8 total, 8 up, 8 in 2024-09-16T08:45:35.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:34 smithi160 ceph-mon[30137]: pgmap v189: 36 pgs: 11 creating+peering, 2 active+clean, 23 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:35.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:34 smithi160 ceph-mon[40028]: from='client.34373 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T08:45:35.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:34 smithi160 ceph-mon[40028]: from='client.24385 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-16T08:45:35.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:34 smithi160 ceph-mon[40028]: osdmap e55: 8 total, 8 up, 8 in 2024-09-16T08:45:35.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:34 smithi160 ceph-mon[40028]: pgmap v189: 36 pgs: 11 creating+peering, 2 active+clean, 23 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:45:36.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:35 smithi169 ceph-mon[33465]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T08:45:36.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:35 smithi169 ceph-mon[33465]: Cluster is now healthy 2024-09-16T08:45:36.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:35 smithi169 ceph-mon[33465]: osdmap e56: 8 total, 8 up, 8 in 2024-09-16T08:45:36.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3654193317' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-16T08:45:36.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-16T08:45:36.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:35 smithi169 ceph-mon[33465]: from='client.34403 ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-16T08:45:36.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:35 smithi160 ceph-mon[30137]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T08:45:36.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:35 smithi160 ceph-mon[30137]: Cluster is now healthy 2024-09-16T08:45:36.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:35 smithi160 ceph-mon[30137]: osdmap e56: 8 total, 8 up, 8 in 2024-09-16T08:45:36.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3654193317' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-16T08:45:36.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-16T08:45:36.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:35 smithi160 ceph-mon[30137]: from='client.34403 ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-16T08:45:36.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:35 smithi160 ceph-mon[40028]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T08:45:36.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:35 smithi160 ceph-mon[40028]: Cluster is now healthy 2024-09-16T08:45:36.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:35 smithi160 ceph-mon[40028]: osdmap e56: 8 total, 8 up, 8 in 2024-09-16T08:45:36.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3654193317' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-16T08:45:36.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-16T08:45:36.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:35 smithi160 ceph-mon[40028]: from='client.34403 ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-16T08:45:36.432 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.169 --placement '1;smithi169=iscsi.a' 2024-09-16T08:45:36.685 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:45:37.059 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T08:45:37.059 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:36 smithi169 ceph-mon[33465]: from='client.34403 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T08:45:37.059 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:36 smithi169 ceph-mon[33465]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T08:45:37.059 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:36 smithi169 ceph-mon[33465]: pgmap v192: 68 pgs: 20 creating+peering, 24 active+clean, 24 unknown; 578 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:45:37.059 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:36 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:37.150 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.147+0000 7f8d52dd4640 1 -- 172.21.15.169:0/3482649670 >> v1:172.21.15.169:6789/0 conn(0x7f8d4c108f30 legacy=0x7f8d4c10b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:37.150 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.147+0000 7f8d52dd4640 1 -- 172.21.15.169:0/3482649670 shutdown_connections 2024-09-16T08:45:37.150 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.147+0000 7f8d52dd4640 1 -- 172.21.15.169:0/3482649670 >> 172.21.15.169:0/3482649670 conn(0x7f8d4c0fcf80 msgr2=0x7f8d4c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:37.150 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.147+0000 7f8d52dd4640 1 -- 172.21.15.169:0/3482649670 shutdown_connections 2024-09-16T08:45:37.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.147+0000 7f8d52dd4640 1 -- 172.21.15.169:0/3482649670 wait complete. 2024-09-16T08:45:37.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.147+0000 7f8d52dd4640 1 Processor -- start 2024-09-16T08:45:37.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.147+0000 7f8d52dd4640 1 -- start start 2024-09-16T08:45:37.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.147+0000 7f8d52dd4640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f8d4c1a4fd0 con 0x7f8d4c06be60 2024-09-16T08:45:37.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.147+0000 7f8d52dd4640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f8d4c1a61b0 con 0x7f8d4c108f30 2024-09-16T08:45:37.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.147+0000 7f8d52dd4640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f8d4c1a7390 con 0x7f8d4c10cb00 2024-09-16T08:45:37.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.147+0000 7f8d43fff640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f8d4c108f30 0x7f8d4c1a0180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:42522/0 (socket says 172.21.15.169:42522) 2024-09-16T08:45:37.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.147+0000 7f8d43fff640 1 -- 172.21.15.169:0/2035671836 learned_addr learned my addr 172.21.15.169:0/2035671836 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:45:37.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3296451494 0 0) 0x7f8d4c1a61b0 con 0x7f8d4c108f30 2024-09-16T08:45:37.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d34003520 con 0x7f8d4c108f30 2024-09-16T08:45:37.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2164920903 0 0) 0x7f8d4c1a4fd0 con 0x7f8d4c06be60 2024-09-16T08:45:37.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d4c1a61b0 con 0x7f8d4c06be60 2024-09-16T08:45:37.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3843794311 0 0) 0x7f8d4c1a7390 con 0x7f8d4c10cb00 2024-09-16T08:45:37.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d4c1a4fd0 con 0x7f8d4c10cb00 2024-09-16T08:45:37.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 276944223 0 0) 0x7f8d34003520 con 0x7f8d4c108f30 2024-09-16T08:45:37.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8d4c1a7390 con 0x7f8d4c108f30 2024-09-16T08:45:37.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f8d3c0014b0 con 0x7f8d4c108f30 2024-09-16T08:45:37.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3225206543 0 0) 0x7f8d4c1a61b0 con 0x7f8d4c06be60 2024-09-16T08:45:37.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8d34003520 con 0x7f8d4c06be60 2024-09-16T08:45:37.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f8d48001930 con 0x7f8d4c06be60 2024-09-16T08:45:37.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 <== mon.1 v1:172.21.15.169:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1238012113 0 0) 0x7f8d4c1a7390 con 0x7f8d4c108f30 2024-09-16T08:45:37.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 >> v1:172.21.15.160:6790/0 conn(0x7f8d4c10cb00 legacy=0x7f8d4c1a38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:37.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 >> v1:172.21.15.160:6789/0 conn(0x7f8d4c06be60 legacy=0x7f8d4c199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:37.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4c1a8570 con 0x7f8d4c108f30 2024-09-16T08:45:37.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d52dd4640 1 -- 172.21.15.169:0/2035671836 --> v1:172.21.15.169:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4c1a5200 con 0x7f8d4c108f30 2024-09-16T08:45:37.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 <== mon.1 v1:172.21.15.169:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f8d3c001960 con 0x7f8d4c108f30 2024-09-16T08:45:37.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 <== mon.1 v1:172.21.15.169:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f8d3c004fe0 con 0x7f8d4c108f30 2024-09-16T08:45:37.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.148+0000 7f8d52dd4640 1 -- 172.21.15.169:0/2035671836 --> v1:172.21.15.169:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f8d4c1a5830 con 0x7f8d4c108f30 2024-09-16T08:45:37.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.149+0000 7f8d52dd4640 1 -- 172.21.15.169:0/2035671836 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d4c111c70 con 0x7f8d4c108f30 2024-09-16T08:45:37.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.150+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 <== mon.1 v1:172.21.15.169:6789/0 7 ==== mgrmap(e 15) ==== 103045+0+0 (unknown 4003825774 0 0) 0x7f8d3c004fc0 con 0x7f8d4c108f30 2024-09-16T08:45:37.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.151+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 <== mon.1 v1:172.21.15.169:6789/0 8 ==== osd_map(58..58 src has 1..58) ==== 5549+0+0 (unknown 4266789994 0 0) 0x7f8d3c098f00 con 0x7f8d4c108f30 2024-09-16T08:45:37.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.155+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 <== mon.1 v1:172.21.15.169:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f8d3c0604e0 con 0x7f8d4c108f30 2024-09-16T08:45:37.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T08:45:37.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:36 smithi160 ceph-mon[30137]: from='client.34403 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T08:45:37.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:36 smithi160 ceph-mon[30137]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T08:45:37.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:36 smithi160 ceph-mon[30137]: pgmap v192: 68 pgs: 20 creating+peering, 24 active+clean, 24 unknown; 578 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:45:37.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:36 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:37.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T08:45:37.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:36 smithi160 ceph-mon[40028]: from='client.34403 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T08:45:37.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:36 smithi160 ceph-mon[40028]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T08:45:37.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:36 smithi160 ceph-mon[40028]: pgmap v192: 68 pgs: 20 creating+peering, 24 active+clean, 24 unknown; 578 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:45:37.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:36 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:37.342 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.341+0000 7f8d52dd4640 1 -- 172.21.15.169:0/2035671836 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.169", "placement": "1;smithi169=iscsi.a", "target": ["mon-mgr", ""]}) -- 0x7f8d4c1a5e50 con 0x7f8d3407b8b0 2024-09-16T08:45:37.350 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.350+0000 7f8d41ffb640 1 -- 172.21.15.169:0/2035671836 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+35 (unknown 0 0 803663096) 0x7f8d4c1a5e50 con 0x7f8d3407b8b0 2024-09-16T08:45:37.351 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled iscsi.datapool update... 2024-09-16T08:45:37.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.352+0000 7f8d52dd4640 1 -- 172.21.15.169:0/2035671836 >> v1:172.21.15.160:6800/1286008384 conn(0x7f8d3407b8b0 legacy=0x7f8d3407dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:37.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.352+0000 7f8d52dd4640 1 -- 172.21.15.169:0/2035671836 >> v1:172.21.15.169:6789/0 conn(0x7f8d4c108f30 legacy=0x7f8d4c1a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:37.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.352+0000 7f8d52dd4640 1 -- 172.21.15.169:0/2035671836 shutdown_connections 2024-09-16T08:45:37.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.352+0000 7f8d52dd4640 1 -- 172.21.15.169:0/2035671836 >> 172.21.15.169:0/2035671836 conn(0x7f8d4c0fcf80 msgr2=0x7f8d4c10efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:37.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.352+0000 7f8d52dd4640 1 -- 172.21.15.169:0/2035671836 shutdown_connections 2024-09-16T08:45:37.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:37.352+0000 7f8d52dd4640 1 -- 172.21.15.169:0/2035671836 wait complete. 2024-09-16T08:45:37.946 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:37 smithi169 ceph-mon[33465]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T08:45:37.946 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3654193317' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-16T08:45:37.946 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-16T08:45:37.946 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:37 smithi169 ceph-mon[33465]: from='client.34403 ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-16T08:45:37.946 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:37 smithi169 ceph-mon[33465]: from='client.24430 v1:172.21.15.169:0/2035671836' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.169", "placement": "1;smithi169=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:37.946 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:37 smithi169 ceph-mon[33465]: Saving service iscsi.datapool spec with placement smithi169=iscsi.a;count:1 2024-09-16T08:45:37.946 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:37 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:37.973 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-16T08:45:37.974 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:45:37.974 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-16T08:45:38.003 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T08:45:38.003 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-16T08:45:38.030 DEBUG:teuthology.orchestra.run.smithi169:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@iscsi.iscsi.a.service 2024-09-16T08:45:38.072 INFO:tasks.cephadm:Adding prometheus.a on smithi169 2024-09-16T08:45:38.073 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch apply prometheus '1;smithi169=a' 2024-09-16T08:45:38.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:37 smithi160 ceph-mon[30137]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T08:45:38.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3654193317' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-16T08:45:38.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-16T08:45:38.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:37 smithi160 ceph-mon[30137]: from='client.34403 ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-16T08:45:38.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:37 smithi160 ceph-mon[30137]: from='client.24430 v1:172.21.15.169:0/2035671836' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.169", "placement": "1;smithi169=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:38.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:37 smithi160 ceph-mon[30137]: Saving service iscsi.datapool spec with placement smithi169=iscsi.a;count:1 2024-09-16T08:45:38.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:37 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:38.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:37 smithi160 ceph-mon[40028]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T08:45:38.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3654193317' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-16T08:45:38.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-16T08:45:38.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:37 smithi160 ceph-mon[40028]: from='client.34403 ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-16T08:45:38.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:37 smithi160 ceph-mon[40028]: from='client.24430 v1:172.21.15.169:0/2035671836' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.169", "placement": "1;smithi169=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:38.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:37 smithi160 ceph-mon[40028]: Saving service iscsi.datapool spec with placement smithi169=iscsi.a;count:1 2024-09-16T08:45:38.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:37 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:38.356 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:45:38.801 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.798+0000 7ff4bb38e640 1 -- 172.21.15.169:0/4233619620 >> v1:172.21.15.169:6789/0 conn(0x7ff4b4104ae0 legacy=0x7ff4b4106f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:38.801 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.798+0000 7ff4bb38e640 1 -- 172.21.15.169:0/4233619620 shutdown_connections 2024-09-16T08:45:38.801 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.798+0000 7ff4bb38e640 1 -- 172.21.15.169:0/4233619620 >> 172.21.15.169:0/4233619620 conn(0x7ff4b40fc710 msgr2=0x7ff4b40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:38.801 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.799+0000 7ff4bb38e640 1 -- 172.21.15.169:0/4233619620 shutdown_connections 2024-09-16T08:45:38.801 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.799+0000 7ff4bb38e640 1 -- 172.21.15.169:0/4233619620 wait complete. 2024-09-16T08:45:38.801 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.799+0000 7ff4bb38e640 1 Processor -- start 2024-09-16T08:45:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.799+0000 7ff4bb38e640 1 -- start start 2024-09-16T08:45:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.799+0000 7ff4bb38e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff4b41a96b0 con 0x7ff4b4100ae0 2024-09-16T08:45:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.799+0000 7ff4bb38e640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff4b41aa890 con 0x7ff4b41076c0 2024-09-16T08:45:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.799+0000 7ff4bb38e640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff4b41aba70 con 0x7ff4b4104ae0 2024-09-16T08:45:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.799+0000 7ff4b912a640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7ff4b4100ae0 0x7ff4b419e620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.169:50364/0 (socket says 172.21.15.169:50364) 2024-09-16T08:45:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.799+0000 7ff4b992b640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7ff4b41076c0 0x7ff4b41a7f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:42550/0 (socket says 172.21.15.169:42550) 2024-09-16T08:45:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.799+0000 7ff4b912a640 1 -- 172.21.15.169:0/4196291018 learned_addr learned my addr 172.21.15.169:0/4196291018 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:45:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2070909894 0 0) 0x7ff4b41aa890 con 0x7ff4b41076c0 2024-09-16T08:45:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4a0003520 con 0x7ff4b41076c0 2024-09-16T08:45:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1612206170 0 0) 0x7ff4b41aba70 con 0x7ff4b4104ae0 2024-09-16T08:45:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4b41aa890 con 0x7ff4b4104ae0 2024-09-16T08:45:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2286477157 0 0) 0x7ff4b41a96b0 con 0x7ff4b4100ae0 2024-09-16T08:45:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4b41aba70 con 0x7ff4b4100ae0 2024-09-16T08:45:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 476042738 0 0) 0x7ff4a0003520 con 0x7ff4b41076c0 2024-09-16T08:45:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff4b41a96b0 con 0x7ff4b41076c0 2024-09-16T08:45:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ff4ac0018f0 con 0x7ff4b41076c0 2024-09-16T08:45:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4030369116 0 0) 0x7ff4b41aa890 con 0x7ff4b4104ae0 2024-09-16T08:45:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff4a0003520 con 0x7ff4b4104ae0 2024-09-16T08:45:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3746469477 0 0) 0x7ff4b41aba70 con 0x7ff4b4100ae0 2024-09-16T08:45:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff4b41aa890 con 0x7ff4b4100ae0 2024-09-16T08:45:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ff4a40014b0 con 0x7ff4b4104ae0 2024-09-16T08:45:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ff4b0001c30 con 0x7ff4b4100ae0 2024-09-16T08:45:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 <== mon.1 v1:172.21.15.169:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 141192690 0 0) 0x7ff4b41a96b0 con 0x7ff4b41076c0 2024-09-16T08:45:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 >> v1:172.21.15.160:6790/0 conn(0x7ff4b4104ae0 legacy=0x7ff4b41a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 >> v1:172.21.15.160:6789/0 conn(0x7ff4b4100ae0 legacy=0x7ff4b419e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b41acc50 con 0x7ff4b41076c0 2024-09-16T08:45:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4bb38e640 1 -- 172.21.15.169:0/4196291018 --> v1:172.21.15.169:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b41abca0 con 0x7ff4b41076c0 2024-09-16T08:45:38.804 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4bb38e640 1 -- 172.21.15.169:0/4196291018 --> v1:172.21.15.169:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff4b41ac280 con 0x7ff4b41076c0 2024-09-16T08:45:38.804 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 <== mon.1 v1:172.21.15.169:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7ff4ac003a80 con 0x7ff4b41076c0 2024-09-16T08:45:38.804 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.800+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 <== mon.1 v1:172.21.15.169:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ff4ac005b30 con 0x7ff4b41076c0 2024-09-16T08:45:38.805 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.801+0000 7ff4bb38e640 1 -- 172.21.15.169:0/4196291018 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4980050b0 con 0x7ff4b41076c0 2024-09-16T08:45:38.805 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.802+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 <== mon.1 v1:172.21.15.169:6789/0 7 ==== mgrmap(e 15) ==== 103045+0+0 (unknown 4003825774 0 0) 0x7ff4ac006dc0 con 0x7ff4b41076c0 2024-09-16T08:45:38.805 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.802+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 <== mon.1 v1:172.21.15.169:6789/0 8 ==== osd_map(59..59 src has 1..59) ==== 5560+0+0 (unknown 366562860 0 0) 0x7ff4ac098f60 con 0x7ff4b41076c0 2024-09-16T08:45:38.807 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:38.806+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 <== mon.1 v1:172.21.15.169:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7ff4ac0603f0 con 0x7ff4b41076c0 2024-09-16T08:45:39.004 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:39.003+0000 7ff4bb38e640 1 -- 172.21.15.169:0/4196291018 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi169=a", "target": ["mon-mgr", ""]}) -- 0x7ff498000bc0 con 0x7ff4a007bc30 2024-09-16T08:45:39.011 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:39.010+0000 7ff4aa7fc640 1 -- 172.21.15.169:0/4196291018 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+31 (unknown 0 0 1342662408) 0x7ff498000bc0 con 0x7ff4a007bc30 2024-09-16T08:45:39.011 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled prometheus update... 2024-09-16T08:45:39.013 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:39.013+0000 7ff4bb38e640 1 -- 172.21.15.169:0/4196291018 >> v1:172.21.15.160:6800/1286008384 conn(0x7ff4a007bc30 legacy=0x7ff4a007e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:39.013 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:39.013+0000 7ff4bb38e640 1 -- 172.21.15.169:0/4196291018 >> v1:172.21.15.169:6789/0 conn(0x7ff4b41076c0 legacy=0x7ff4b41a7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:39.013 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:39.013+0000 7ff4bb38e640 1 -- 172.21.15.169:0/4196291018 shutdown_connections 2024-09-16T08:45:39.014 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:39.013+0000 7ff4bb38e640 1 -- 172.21.15.169:0/4196291018 >> 172.21.15.169:0/4196291018 conn(0x7ff4b40fc710 msgr2=0x7ff4b4106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:39.014 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:39.013+0000 7ff4bb38e640 1 -- 172.21.15.169:0/4196291018 shutdown_connections 2024-09-16T08:45:39.014 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:39.013+0000 7ff4bb38e640 1 -- 172.21.15.169:0/4196291018 wait complete. 2024-09-16T08:45:39.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T08:45:39.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:38 smithi169 ceph-mon[33465]: from='client.34403 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T08:45:39.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:38 smithi169 ceph-mon[33465]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T08:45:39.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:38 smithi169 ceph-mon[33465]: pgmap v195: 100 pgs: 25 creating+peering, 26 active+clean, 49 unknown; 578 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:45:39.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T08:45:39.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:38 smithi160 ceph-mon[30137]: from='client.34403 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T08:45:39.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:38 smithi160 ceph-mon[30137]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T08:45:39.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:38 smithi160 ceph-mon[30137]: pgmap v195: 100 pgs: 25 creating+peering, 26 active+clean, 49 unknown; 578 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:45:39.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T08:45:39.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:38 smithi160 ceph-mon[40028]: from='client.34403 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T08:45:39.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:38 smithi160 ceph-mon[40028]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T08:45:39.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:38 smithi160 ceph-mon[40028]: pgmap v195: 100 pgs: 25 creating+peering, 26 active+clean, 49 unknown; 578 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:45:39.648 DEBUG:teuthology.orchestra.run.smithi169:prometheus.a> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@prometheus.a.service 2024-09-16T08:45:39.650 INFO:tasks.cephadm:Adding node-exporter.a on smithi160 2024-09-16T08:45:39.650 INFO:tasks.cephadm:Adding node-exporter.b on smithi169 2024-09-16T08:45:39.650 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch apply node-exporter '2;smithi160=a;smithi169=b' 2024-09-16T08:45:39.940 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:45:40.060 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:39 smithi169 ceph-mon[33465]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T08:45:40.061 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3654193317' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-16T08:45:40.061 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:39 smithi169 ceph-mon[33465]: from='client.34403 ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-16T08:45:40.061 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-16T08:45:40.061 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:39 smithi169 ceph-mon[33465]: from='client.24436 v1:172.21.15.169:0/4196291018' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi169=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:40.061 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:39 smithi169 ceph-mon[33465]: Saving service prometheus spec with placement smithi169=a;count:1 2024-09-16T08:45:40.061 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:39 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:40.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:39 smithi160 ceph-mon[30137]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T08:45:40.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3654193317' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-16T08:45:40.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:39 smithi160 ceph-mon[30137]: from='client.34403 ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-16T08:45:40.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-16T08:45:40.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:39 smithi160 ceph-mon[30137]: from='client.24436 v1:172.21.15.169:0/4196291018' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi169=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:40.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:39 smithi160 ceph-mon[30137]: Saving service prometheus spec with placement smithi169=a;count:1 2024-09-16T08:45:40.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:39 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:40.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:39 smithi160 ceph-mon[40028]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T08:45:40.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3654193317' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-16T08:45:40.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:39 smithi160 ceph-mon[40028]: from='client.34403 ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-16T08:45:40.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-16T08:45:40.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:39 smithi160 ceph-mon[40028]: from='client.24436 v1:172.21.15.169:0/4196291018' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi169=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:40.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:39 smithi160 ceph-mon[40028]: Saving service prometheus spec with placement smithi169=a;count:1 2024-09-16T08:45:40.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:39 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:40.382 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.372+0000 7f298f92d640 1 -- 172.21.15.169:0/3030079876 >> v1:172.21.15.169:6789/0 conn(0x7f298806bff0 legacy=0x7f29881053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:40.382 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.372+0000 7f298f92d640 1 -- 172.21.15.169:0/3030079876 shutdown_connections 2024-09-16T08:45:40.382 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.372+0000 7f298f92d640 1 -- 172.21.15.169:0/3030079876 >> 172.21.15.169:0/3030079876 conn(0x7f29880fcfb0 msgr2=0x7f29880ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:40.382 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.372+0000 7f298f92d640 1 -- 172.21.15.169:0/3030079876 shutdown_connections 2024-09-16T08:45:40.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.372+0000 7f298f92d640 1 -- 172.21.15.169:0/3030079876 wait complete. 2024-09-16T08:45:40.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.372+0000 7f298f92d640 1 Processor -- start 2024-09-16T08:45:40.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f298f92d640 1 -- start start 2024-09-16T08:45:40.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f298f92d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f29881a4ff0 con 0x7f298810a8f0 2024-09-16T08:45:40.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f298f92d640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f29881a61f0 con 0x7f298806bff0 2024-09-16T08:45:40.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f298f92d640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f29881a73f0 con 0x7f2988106b70 2024-09-16T08:45:40.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f298d6c9640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f298806bff0 0x7f2988102a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:42566/0 (socket says 172.21.15.169:42566) 2024-09-16T08:45:40.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f298d6c9640 1 -- 172.21.15.169:0/3586314419 learned_addr learned my addr 172.21.15.169:0/3586314419 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:45:40.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3858768676 0 0) 0x7f29881a61f0 con 0x7f298806bff0 2024-09-16T08:45:40.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2974003520 con 0x7f298806bff0 2024-09-16T08:45:40.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3330630966 0 0) 0x7f29881a4ff0 con 0x7f298810a8f0 2024-09-16T08:45:40.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29881a61f0 con 0x7f298810a8f0 2024-09-16T08:45:40.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3015107856 0 0) 0x7f29881a73f0 con 0x7f2988106b70 2024-09-16T08:45:40.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29881a4ff0 con 0x7f2988106b70 2024-09-16T08:45:40.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2942060989 0 0) 0x7f2974003520 con 0x7f298806bff0 2024-09-16T08:45:40.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f29881a73f0 con 0x7f298806bff0 2024-09-16T08:45:40.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f2984001e70 con 0x7f298806bff0 2024-09-16T08:45:40.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 <== mon.1 v1:172.21.15.169:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1954644905 0 0) 0x7f29881a73f0 con 0x7f298806bff0 2024-09-16T08:45:40.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 >> v1:172.21.15.160:6790/0 conn(0x7f2988106b70 legacy=0x7f298819fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:40.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 >> v1:172.21.15.160:6789/0 conn(0x7f298810a8f0 legacy=0x7f29881a36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:40.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29881a8690 con 0x7f298806bff0 2024-09-16T08:45:40.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.373+0000 7f298f92d640 1 -- 172.21.15.169:0/3586314419 --> v1:172.21.15.169:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f29881a75c0 con 0x7f298806bff0 2024-09-16T08:45:40.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.374+0000 7f298f92d640 1 -- 172.21.15.169:0/3586314419 --> v1:172.21.15.169:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f29881a7bf0 con 0x7f298806bff0 2024-09-16T08:45:40.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.374+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 <== mon.1 v1:172.21.15.169:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f29840020a0 con 0x7f298806bff0 2024-09-16T08:45:40.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.374+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 <== mon.1 v1:172.21.15.169:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f2984005120 con 0x7f298806bff0 2024-09-16T08:45:40.385 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.374+0000 7f298f92d640 1 -- 172.21.15.169:0/3586314419 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f296c0050b0 con 0x7f298806bff0 2024-09-16T08:45:40.385 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.375+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 <== mon.1 v1:172.21.15.169:6789/0 7 ==== mgrmap(e 15) ==== 103045+0+0 (unknown 4003825774 0 0) 0x7f2984003b80 con 0x7f298806bff0 2024-09-16T08:45:40.385 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.376+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 <== mon.1 v1:172.21.15.169:6789/0 8 ==== osd_map(61..61 src has 1..61) ==== 5926+0+0 (unknown 2751307097 0 0) 0x7f29840996f0 con 0x7f298806bff0 2024-09-16T08:45:40.385 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.380+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 <== mon.1 v1:172.21.15.169:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f2984060a50 con 0x7f298806bff0 2024-09-16T08:45:40.570 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.569+0000 7f298f92d640 1 -- 172.21.15.169:0/3586314419 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi160=a;smithi169=b", "target": ["mon-mgr", ""]}) -- 0x7f296c000bc0 con 0x7f297407baf0 2024-09-16T08:45:40.577 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.577+0000 7f297e7fc640 1 -- 172.21.15.169:0/3586314419 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+34 (unknown 0 0 240551134) 0x7f296c000bc0 con 0x7f297407baf0 2024-09-16T08:45:40.578 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled node-exporter update... 2024-09-16T08:45:40.580 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.579+0000 7f298f92d640 1 -- 172.21.15.169:0/3586314419 >> v1:172.21.15.160:6800/1286008384 conn(0x7f297407baf0 legacy=0x7f297407dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:40.580 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.579+0000 7f298f92d640 1 -- 172.21.15.169:0/3586314419 >> v1:172.21.15.169:6789/0 conn(0x7f298806bff0 legacy=0x7f2988102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:40.580 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.579+0000 7f298f92d640 1 -- 172.21.15.169:0/3586314419 shutdown_connections 2024-09-16T08:45:40.580 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.579+0000 7f298f92d640 1 -- 172.21.15.169:0/3586314419 >> 172.21.15.169:0/3586314419 conn(0x7f29880fcfb0 msgr2=0x7f29880ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:40.580 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.579+0000 7f298f92d640 1 -- 172.21.15.169:0/3586314419 shutdown_connections 2024-09-16T08:45:40.580 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:40.579+0000 7f298f92d640 1 -- 172.21.15.169:0/3586314419 wait complete. 2024-09-16T08:45:41.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[30137]: from='client.34403 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T08:45:41.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T08:45:41.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[30137]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T08:45:41.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3654193317' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-16T08:45:41.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-16T08:45:41.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[30137]: from='client.34403 ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-16T08:45:41.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[30137]: pgmap v198: 132 pgs: 30 creating+peering, 82 active+clean, 20 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-16T08:45:41.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[30137]: from='client.24442 v1:172.21.15.169:0/3586314419' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi160=a;smithi169=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:41.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[30137]: Saving service node-exporter spec with placement smithi160=a;smithi169=b;count:2 2024-09-16T08:45:41.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:41.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[40028]: from='client.34403 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T08:45:41.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T08:45:41.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[40028]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T08:45:41.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3654193317' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-16T08:45:41.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-16T08:45:41.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[40028]: from='client.34403 ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-16T08:45:41.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[40028]: pgmap v198: 132 pgs: 30 creating+peering, 82 active+clean, 20 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-16T08:45:41.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[40028]: from='client.24442 v1:172.21.15.169:0/3586314419' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi160=a;smithi169=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:41.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[40028]: Saving service node-exporter spec with placement smithi160=a;smithi169=b;count:2 2024-09-16T08:45:41.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:40 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:41.221 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:40 smithi169 ceph-mon[33465]: from='client.34403 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T08:45:41.221 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T08:45:41.221 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:40 smithi169 ceph-mon[33465]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T08:45:41.221 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3654193317' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-16T08:45:41.221 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-16T08:45:41.221 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:40 smithi169 ceph-mon[33465]: from='client.34403 ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-16T08:45:41.222 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:40 smithi169 ceph-mon[33465]: pgmap v198: 132 pgs: 30 creating+peering, 82 active+clean, 20 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-16T08:45:41.222 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:40 smithi169 ceph-mon[33465]: from='client.24442 v1:172.21.15.169:0/3586314419' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi160=a;smithi169=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:41.222 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:40 smithi169 ceph-mon[33465]: Saving service node-exporter spec with placement smithi160=a;smithi169=b;count:2 2024-09-16T08:45:41.222 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:40 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:41.250 DEBUG:teuthology.orchestra.run.smithi160:node-exporter.a> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@node-exporter.a.service 2024-09-16T08:45:41.252 DEBUG:teuthology.orchestra.run.smithi169:node-exporter.b> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@node-exporter.b.service 2024-09-16T08:45:41.255 INFO:tasks.cephadm:Adding alertmanager.a on smithi160 2024-09-16T08:45:41.255 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch apply alertmanager '1;smithi160=a' 2024-09-16T08:45:41.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T08:45:41.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[30137]: from='client.34403 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T08:45:41.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[30137]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T08:45:41.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:41.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:41.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:41.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-16T08:45:41.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T08:45:41.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:41.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[30137]: Deploying daemon iscsi.iscsi.a on smithi169 2024-09-16T08:45:41.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T08:45:41.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[40028]: from='client.34403 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T08:45:41.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[40028]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T08:45:41.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:41.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:41.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:41.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-16T08:45:41.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T08:45:41.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:41.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:41 smithi160 ceph-mon[40028]: Deploying daemon iscsi.iscsi.a on smithi169 2024-09-16T08:45:42.092 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:45:42.212 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4093404084' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T08:45:42.212 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:41 smithi169 ceph-mon[33465]: from='client.34403 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T08:45:42.212 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:41 smithi169 ceph-mon[33465]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T08:45:42.212 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:42.212 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:42.212 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:42.213 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-16T08:45:42.213 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T08:45:42.213 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:41 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:45:42.213 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:41 smithi169 ceph-mon[33465]: Deploying daemon iscsi.iscsi.a on smithi169 2024-09-16T08:45:43.155 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:43 smithi169 ceph-mon[33465]: pgmap v200: 132 pgs: 21 creating+peering, 100 active+clean, 11 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-16T08:45:43.156 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:43 smithi169 systemd[1]: Starting Ceph iscsi.iscsi.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T08:45:43.315 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.311+0000 7f863b9d2640 1 -- 172.21.15.169:0/938954010 >> v1:172.21.15.169:6789/0 conn(0x7f86341097b0 legacy=0x7f863410bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:43.315 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.311+0000 7f863b9d2640 1 -- 172.21.15.169:0/938954010 shutdown_connections 2024-09-16T08:45:43.316 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.311+0000 7f863b9d2640 1 -- 172.21.15.169:0/938954010 >> 172.21.15.169:0/938954010 conn(0x7f86340fcf70 msgr2=0x7f86340ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:43.316 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.311+0000 7f863b9d2640 1 -- 172.21.15.169:0/938954010 shutdown_connections 2024-09-16T08:45:43.316 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.311+0000 7f863b9d2640 1 -- 172.21.15.169:0/938954010 wait complete. 2024-09-16T08:45:43.316 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.311+0000 7f863b9d2640 1 Processor -- start 2024-09-16T08:45:43.316 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.311+0000 7f863b9d2640 1 -- start start 2024-09-16T08:45:43.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f863b9d2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f86341a2f90 con 0x7f86341097b0 2024-09-16T08:45:43.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f863b9d2640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f86341a4170 con 0x7f8634105b20 2024-09-16T08:45:43.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f863b9d2640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f86341a5350 con 0x7f86341016a0 2024-09-16T08:45:43.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f8638f6d640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f8634105b20 0x7f863419e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:42582/0 (socket says 172.21.15.169:42582) 2024-09-16T08:45:43.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f8638f6d640 1 -- 172.21.15.169:0/1447176229 learned_addr learned my addr 172.21.15.169:0/1447176229 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:45:43.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2832523454 0 0) 0x7f86341a4170 con 0x7f8634105b20 2024-09-16T08:45:43.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8620003520 con 0x7f8634105b20 2024-09-16T08:45:43.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1968459168 0 0) 0x7f86341a5350 con 0x7f86341016a0 2024-09-16T08:45:43.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86341a4170 con 0x7f86341016a0 2024-09-16T08:45:43.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1535374061 0 0) 0x7f86341a2f90 con 0x7f86341097b0 2024-09-16T08:45:43.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86341a5350 con 0x7f86341097b0 2024-09-16T08:45:43.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 43793146 0 0) 0x7f8620003520 con 0x7f8634105b20 2024-09-16T08:45:43.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f86341a2f90 con 0x7f8634105b20 2024-09-16T08:45:43.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f862c001d70 con 0x7f8634105b20 2024-09-16T08:45:43.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 767277076 0 0) 0x7f86341a4170 con 0x7f86341016a0 2024-09-16T08:45:43.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8620003520 con 0x7f86341016a0 2024-09-16T08:45:43.320 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f86300018a0 con 0x7f86341016a0 2024-09-16T08:45:43.320 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 <== mon.1 v1:172.21.15.169:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 614792921 0 0) 0x7f86341a2f90 con 0x7f8634105b20 2024-09-16T08:45:43.320 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 >> v1:172.21.15.160:6790/0 conn(0x7f86341016a0 legacy=0x7f8634197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:43.320 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 >> v1:172.21.15.160:6789/0 conn(0x7f86341097b0 legacy=0x7f86341a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:43.321 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86341a6530 con 0x7f8634105b20 2024-09-16T08:45:43.321 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f863b9d2640 1 -- 172.21.15.169:0/1447176229 --> v1:172.21.15.169:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f86341a5580 con 0x7f8634105b20 2024-09-16T08:45:43.321 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f863b9d2640 1 -- 172.21.15.169:0/1447176229 --> v1:172.21.15.169:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f86341a5bb0 con 0x7f8634105b20 2024-09-16T08:45:43.321 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.312+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 <== mon.1 v1:172.21.15.169:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f862c001ff0 con 0x7f8634105b20 2024-09-16T08:45:43.321 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.313+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 <== mon.1 v1:172.21.15.169:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f862c005fd0 con 0x7f8634105b20 2024-09-16T08:45:43.322 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.313+0000 7f863b9d2640 1 -- 172.21.15.169:0/1447176229 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86180050b0 con 0x7f8634105b20 2024-09-16T08:45:43.322 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.314+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 <== mon.1 v1:172.21.15.169:6789/0 7 ==== mgrmap(e 15) ==== 103045+0+0 (unknown 4003825774 0 0) 0x7f862c003c40 con 0x7f8634105b20 2024-09-16T08:45:43.322 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.315+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 <== mon.1 v1:172.21.15.169:6789/0 8 ==== osd_map(62..62 src has 1..62) ==== 5942+0+0 (unknown 2364203128 0 0) 0x7f862c0993f0 con 0x7f8634105b20 2024-09-16T08:45:43.322 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.319+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 <== mon.1 v1:172.21.15.169:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f862c060840 con 0x7f8634105b20 2024-09-16T08:45:43.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:43 smithi160 ceph-mon[40028]: pgmap v200: 132 pgs: 21 creating+peering, 100 active+clean, 11 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-16T08:45:43.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:43 smithi160 ceph-mon[30137]: pgmap v200: 132 pgs: 21 creating+peering, 100 active+clean, 11 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-16T08:45:43.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.519+0000 7f863b9d2640 1 -- 172.21.15.169:0/1447176229 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi160=a", "target": ["mon-mgr", ""]}) -- 0x7f8618000bc0 con 0x7f862007b820 2024-09-16T08:45:43.529 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled alertmanager update... 2024-09-16T08:45:43.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.528+0000 7f862a7fc640 1 -- 172.21.15.169:0/1447176229 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+33 (unknown 0 0 1850065467) 0x7f8618000bc0 con 0x7f862007b820 2024-09-16T08:45:43.531 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.531+0000 7f863b9d2640 1 -- 172.21.15.169:0/1447176229 >> v1:172.21.15.160:6800/1286008384 conn(0x7f862007b820 legacy=0x7f862007dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:43.531 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.531+0000 7f863b9d2640 1 -- 172.21.15.169:0/1447176229 >> v1:172.21.15.169:6789/0 conn(0x7f8634105b20 legacy=0x7f863419e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:43.531 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.531+0000 7f863b9d2640 1 -- 172.21.15.169:0/1447176229 shutdown_connections 2024-09-16T08:45:43.532 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.531+0000 7f863b9d2640 1 -- 172.21.15.169:0/1447176229 >> 172.21.15.169:0/1447176229 conn(0x7f86340fcf70 msgr2=0x7f863410cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:43.532 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.531+0000 7f863b9d2640 1 -- 172.21.15.169:0/1447176229 shutdown_connections 2024-09-16T08:45:43.532 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:43.531+0000 7f863b9d2640 1 -- 172.21.15.169:0/1447176229 wait complete. 2024-09-16T08:45:43.850 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:43 smithi169 podman[60801]: 2024-09-16 08:45:43.429860856 +0000 UTC m=+0.016065538 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T08:45:43.850 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:43 smithi169 podman[60801]: 2024-09-16 08:45:43.560158467 +0000 UTC m=+0.146363154 container create 4f95bb703dd63dc4d64177962172c1812c2f3ddae0237c3d6efa94fd1eaad0e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T08:45:44.218 DEBUG:teuthology.orchestra.run.smithi160:alertmanager.a> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@alertmanager.a.service 2024-09-16T08:45:44.221 INFO:tasks.cephadm:Adding grafana.a on smithi169 2024-09-16T08:45:44.221 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph orch apply grafana '1;smithi169=a' 2024-09-16T08:45:44.449 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 podman[60801]: 2024-09-16 08:45:44.34165619 +0000 UTC m=+0.927860875 container init 4f95bb703dd63dc4d64177962172c1812c2f3ddae0237c3d6efa94fd1eaad0e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T08:45:44.449 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 podman[60801]: 2024-09-16 08:45:44.347019129 +0000 UTC m=+0.933223819 container start 4f95bb703dd63dc4d64177962172c1812c2f3ddae0237c3d6efa94fd1eaad0e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T08:45:44.449 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 bash[60801]: 4f95bb703dd63dc4d64177962172c1812c2f3ddae0237c3d6efa94fd1eaad0e3 2024-09-16T08:45:44.449 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 systemd[1]: Started Ceph iscsi.iscsi.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:45:44.519 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:45:44.795 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug Started the configuration object watcher 2024-09-16T08:45:44.795 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug Checking for config object changes every 1s 2024-09-16T08:45:44.795 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug Processing osd blocklist entries for this node 2024-09-16T08:45:44.795 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-mon[33465]: from='client.24463 v1:172.21.15.169:0/1447176229' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi160=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:44.795 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-mon[33465]: Saving service alertmanager spec with placement smithi160=a;count:1 2024-09-16T08:45:44.795 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:44.795 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-mon[33465]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T08:45:44.796 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:44.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:44 smithi160 ceph-mon[40028]: from='client.24463 v1:172.21.15.169:0/1447176229' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi160=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:44.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:44 smithi160 ceph-mon[40028]: Saving service alertmanager spec with placement smithi160=a;count:1 2024-09-16T08:45:44.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:44 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:44.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:44 smithi160 ceph-mon[40028]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T08:45:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:44 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:44 smithi160 ceph-mon[30137]: from='client.24463 v1:172.21.15.169:0/1447176229' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi160=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:44 smithi160 ceph-mon[30137]: Saving service alertmanager spec with placement smithi160=a;count:1 2024-09-16T08:45:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:44 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:44 smithi160 ceph-mon[30137]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T08:45:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:44 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:45.139 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug Reading the configuration object to update local LIO configuration 2024-09-16T08:45:45.140 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug Configuration does not have an entry for this host(smithi169.front.sepia.ceph.com) - nothing to define to LIO 2024-09-16T08:45:45.140 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-16T08:45:45.140 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: * Environment: production 2024-09-16T08:45:45.140 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-16T08:45:45.140 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: Use a production WSGI server instead. 2024-09-16T08:45:45.140 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: * Debug mode: off 2024-09-16T08:45:45.140 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug * Running on all addresses. 2024-09-16T08:45:45.140 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-16T08:45:45.140 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: * Running on all addresses. 2024-09-16T08:45:45.140 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-16T08:45:45.141 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug * Running on http://[fe80::ec4:7aff:fe88:7f4d]:5000/ (Press CTRL+C to quit) 2024-09-16T08:45:45.141 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: * Running on http://[fe80::ec4:7aff:fe88:7f4d]:5000/ (Press CTRL+C to quit) 2024-09-16T08:45:45.448 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.442+0000 7f316de32640 1 -- 172.21.15.169:0/3568744308 >> v1:172.21.15.169:6789/0 conn(0x7f316806bff0 legacy=0x7f31681053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:45.448 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.442+0000 7f316de32640 1 -- 172.21.15.169:0/3568744308 shutdown_connections 2024-09-16T08:45:45.449 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.442+0000 7f316de32640 1 -- 172.21.15.169:0/3568744308 >> 172.21.15.169:0/3568744308 conn(0x7f31680fcfb0 msgr2=0x7f31680ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:45.449 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.442+0000 7f316de32640 1 -- 172.21.15.169:0/3568744308 shutdown_connections 2024-09-16T08:45:45.449 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.442+0000 7f316de32640 1 -- 172.21.15.169:0/3568744308 wait complete. 2024-09-16T08:45:45.449 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.442+0000 7f316de32640 1 Processor -- start 2024-09-16T08:45:45.449 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.442+0000 7f316de32640 1 -- start start 2024-09-16T08:45:45.449 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f316de32640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f31681a2fc0 con 0x7f316810a8f0 2024-09-16T08:45:45.449 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f316de32640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f31681a41a0 con 0x7f316806bff0 2024-09-16T08:45:45.449 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f316de32640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f31681a5380 con 0x7f3168106b70 2024-09-16T08:45:45.449 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31677fe640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f316806bff0 0x7f3168197f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:35986/0 (socket says 172.21.15.169:35986) 2024-09-16T08:45:45.449 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31677fe640 1 -- 172.21.15.169:0/2173371565 learned_addr learned my addr 172.21.15.169:0/2173371565 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:45:45.449 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 322065999 0 0) 0x7f31681a5380 con 0x7f3168106b70 2024-09-16T08:45:45.449 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3154003520 con 0x7f3168106b70 2024-09-16T08:45:45.450 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 997111783 0 0) 0x7f31681a41a0 con 0x7f316806bff0 2024-09-16T08:45:45.450 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31681a5380 con 0x7f316806bff0 2024-09-16T08:45:45.450 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3367861762 0 0) 0x7f31681a2fc0 con 0x7f316810a8f0 2024-09-16T08:45:45.450 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31681a41a0 con 0x7f316810a8f0 2024-09-16T08:45:45.450 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 326776929 0 0) 0x7f3154003520 con 0x7f3168106b70 2024-09-16T08:45:45.450 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f31681a2fc0 con 0x7f3168106b70 2024-09-16T08:45:45.450 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f31500018f0 con 0x7f3168106b70 2024-09-16T08:45:45.450 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3690920395 0 0) 0x7f31681a5380 con 0x7f316806bff0 2024-09-16T08:45:45.450 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3154003520 con 0x7f316806bff0 2024-09-16T08:45:45.450 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1988183376 0 0) 0x7f31681a41a0 con 0x7f316810a8f0 2024-09-16T08:45:45.450 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f31681a5380 con 0x7f316810a8f0 2024-09-16T08:45:45.450 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f31580014b0 con 0x7f316806bff0 2024-09-16T08:45:45.450 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f3160001a80 con 0x7f316810a8f0 2024-09-16T08:45:45.451 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.443+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3403429733 0 0) 0x7f31681a2fc0 con 0x7f3168106b70 2024-09-16T08:45:45.451 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.444+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 >> v1:172.21.15.169:6789/0 conn(0x7f316806bff0 legacy=0x7f3168197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:45.451 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.444+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 >> v1:172.21.15.160:6789/0 conn(0x7f316810a8f0 legacy=0x7f31681a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:45.451 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.444+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31681a6560 con 0x7f3168106b70 2024-09-16T08:45:45.451 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.444+0000 7f316de32640 1 -- 172.21.15.169:0/2173371565 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f31681a4370 con 0x7f3168106b70 2024-09-16T08:45:45.451 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.444+0000 7f316de32640 1 -- 172.21.15.169:0/2173371565 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f31681a49a0 con 0x7f3168106b70 2024-09-16T08:45:45.451 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.444+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f3150003a60 con 0x7f3168106b70 2024-09-16T08:45:45.451 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.444+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f3150004a10 con 0x7f3168106b70 2024-09-16T08:45:45.451 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.445+0000 7f316de32640 1 -- 172.21.15.169:0/2173371565 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31480050b0 con 0x7f3168106b70 2024-09-16T08:45:45.451 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.445+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 15) ==== 103045+0+0 (unknown 4003825774 0 0) 0x7f315001dee0 con 0x7f3168106b70 2024-09-16T08:45:45.451 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.446+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(63..63 src has 1..63) ==== 5942+0+0 (unknown 3139999450 0 0) 0x7f3150097f90 con 0x7f3168106b70 2024-09-16T08:45:45.452 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.450+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f315005f2a0 con 0x7f3168106b70 2024-09-16T08:45:45.677 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.676+0000 7f316de32640 1 -- 172.21.15.169:0/2173371565 --> v1:172.21.15.160:6800/1286008384 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi169=a", "target": ["mon-mgr", ""]}) -- 0x7f3148000bc0 con 0x7f315407bc70 2024-09-16T08:45:45.683 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.683+0000 7f31657fa640 1 -- 172.21.15.169:0/2173371565 <== mgr.14150 v1:172.21.15.160:6800/1286008384 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (unknown 0 0 664801700) 0x7f3148000bc0 con 0x7f315407bc70 2024-09-16T08:45:45.683 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled grafana update... 2024-09-16T08:45:45.685 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.685+0000 7f316de32640 1 -- 172.21.15.169:0/2173371565 >> v1:172.21.15.160:6800/1286008384 conn(0x7f315407bc70 legacy=0x7f315407e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:45.686 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.685+0000 7f316de32640 1 -- 172.21.15.169:0/2173371565 >> v1:172.21.15.160:6790/0 conn(0x7f3168106b70 legacy=0x7f316819e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:45.686 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.685+0000 7f316de32640 1 -- 172.21.15.169:0/2173371565 shutdown_connections 2024-09-16T08:45:45.686 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.685+0000 7f316de32640 1 -- 172.21.15.169:0/2173371565 >> 172.21.15.169:0/2173371565 conn(0x7f31680fcfb0 msgr2=0x7f31680ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:45.686 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.685+0000 7f316de32640 1 -- 172.21.15.169:0/2173371565 shutdown_connections 2024-09-16T08:45:45.686 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:45.685+0000 7f316de32640 1 -- 172.21.15.169:0/2173371565 wait complete. 2024-09-16T08:45:45.695 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:45 smithi169 ceph-mon[33465]: pgmap v201: 132 pgs: 16 creating+peering, 116 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 29 KiB/s rd, 3.3 KiB/s wr, 67 op/s 2024-09-16T08:45:45.695 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:45 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:45.695 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:45 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:45.695 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:45 smithi169 ceph-mon[33465]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-16T08:45:45.695 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:45 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:45.695 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:45 smithi169 ceph-mon[33465]: Deploying daemon prometheus.a on smithi169 2024-09-16T08:45:45.695 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.169:0/3924831049' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-16T08:45:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:45 smithi160 ceph-mon[30137]: pgmap v201: 132 pgs: 16 creating+peering, 116 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 29 KiB/s rd, 3.3 KiB/s wr, 67 op/s 2024-09-16T08:45:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:45 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:45 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:45 smithi160 ceph-mon[30137]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-16T08:45:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:45 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:45 smithi160 ceph-mon[30137]: Deploying daemon prometheus.a on smithi169 2024-09-16T08:45:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.169:0/3924831049' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-16T08:45:45.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:45 smithi160 ceph-mon[40028]: pgmap v201: 132 pgs: 16 creating+peering, 116 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 29 KiB/s rd, 3.3 KiB/s wr, 67 op/s 2024-09-16T08:45:45.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:45 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:45.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:45 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:45.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:45 smithi160 ceph-mon[40028]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-16T08:45:45.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:45 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:45.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:45 smithi160 ceph-mon[40028]: Deploying daemon prometheus.a on smithi169 2024-09-16T08:45:45.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.169:0/3924831049' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-16T08:45:46.491 DEBUG:teuthology.orchestra.run.smithi169:grafana.a> sudo journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@grafana.a.service 2024-09-16T08:45:46.493 INFO:tasks.cephadm:Setting up client nodes... 2024-09-16T08:45:46.493 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T08:45:46.757 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:46.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:46 smithi169 ceph-mon[33465]: from='client.34493 v1:172.21.15.169:0/2173371565' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi169=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:46.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:46 smithi169 ceph-mon[33465]: Saving service grafana spec with placement smithi169=a;count:1 2024-09-16T08:45:46.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:46 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:46.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:46 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:46.878 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:46 smithi160 ceph-mon[30137]: from='client.34493 v1:172.21.15.169:0/2173371565' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi169=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:46.878 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:46 smithi160 ceph-mon[30137]: Saving service grafana spec with placement smithi169=a;count:1 2024-09-16T08:45:46.878 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:46 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:46.879 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:46 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:46.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:46 smithi160 ceph-mon[40028]: from='client.34493 v1:172.21.15.169:0/2173371565' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi169=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:45:46.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:46 smithi160 ceph-mon[40028]: Saving service grafana spec with placement smithi169=a;count:1 2024-09-16T08:45:46.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:46 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:46.879 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:46 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:47.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.260+0000 7f6ad78c2640 1 -- 172.21.15.160:0/295017855 >> v1:172.21.15.160:6790/0 conn(0x7f6ad0104990 legacy=0x7f6ad0106e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:47.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.260+0000 7f6ad78c2640 1 -- 172.21.15.160:0/295017855 shutdown_connections 2024-09-16T08:45:47.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.260+0000 7f6ad78c2640 1 -- 172.21.15.160:0/295017855 >> 172.21.15.160:0/295017855 conn(0x7f6ad00fcf80 msgr2=0x7f6ad00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:47.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.260+0000 7f6ad78c2640 1 -- 172.21.15.160:0/295017855 shutdown_connections 2024-09-16T08:45:47.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.260+0000 7f6ad78c2640 1 -- 172.21.15.160:0/295017855 wait complete. 2024-09-16T08:45:47.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.260+0000 7f6ad78c2640 1 Processor -- start 2024-09-16T08:45:47.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.260+0000 7f6ad78c2640 1 -- start start 2024-09-16T08:45:47.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ad78c2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6ad01a9450 con 0x7f6ad0104990 2024-09-16T08:45:47.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ad78c2640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6ad01aa630 con 0x7f6ad006bff0 2024-09-16T08:45:47.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ad78c2640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6ad01ab810 con 0x7f6ad0110fb0 2024-09-16T08:45:47.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ad5e5f640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f6ad0110fb0 0x7f6ad01a7d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:35116/0 (socket says 172.21.15.160:35116) 2024-09-16T08:45:47.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ad5e5f640 1 -- 172.21.15.160:0/4014402750 learned_addr learned my addr 172.21.15.160:0/4014402750 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:45:47.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1202921298 0 0) 0x7f6ad01a9450 con 0x7f6ad0104990 2024-09-16T08:45:47.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6abc005200 con 0x7f6ad0104990 2024-09-16T08:45:47.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1352218172 0 0) 0x7f6ad01ab810 con 0x7f6ad0110fb0 2024-09-16T08:45:47.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ad01a9450 con 0x7f6ad0110fb0 2024-09-16T08:45:47.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1369636231 0 0) 0x7f6ad01aa630 con 0x7f6ad006bff0 2024-09-16T08:45:47.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ad01ab810 con 0x7f6ad006bff0 2024-09-16T08:45:47.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4016980696 0 0) 0x7f6abc005200 con 0x7f6ad0104990 2024-09-16T08:45:47.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6ad01aa630 con 0x7f6ad0104990 2024-09-16T08:45:47.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f6ac0001630 con 0x7f6ad0104990 2024-09-16T08:45:47.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3166209265 0 0) 0x7f6ad01a9450 con 0x7f6ad0110fb0 2024-09-16T08:45:47.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6abc005200 con 0x7f6ad0110fb0 2024-09-16T08:45:47.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f6ac8001790 con 0x7f6ad0110fb0 2024-09-16T08:45:47.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1944238424 0 0) 0x7f6ad01ab810 con 0x7f6ad006bff0 2024-09-16T08:45:47.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6ad01a9450 con 0x7f6ad006bff0 2024-09-16T08:45:47.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f6acc001830 con 0x7f6ad006bff0 2024-09-16T08:45:47.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1789031772 0 0) 0x7f6ad01aa630 con 0x7f6ad0104990 2024-09-16T08:45:47.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 >> v1:172.21.15.160:6790/0 conn(0x7f6ad0110fb0 legacy=0x7f6ad01a7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:47.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 >> v1:172.21.15.169:6789/0 conn(0x7f6ad006bff0 legacy=0x7f6ad019e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:47.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.261+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad01ac9f0 con 0x7f6ad0104990 2024-09-16T08:45:47.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.262+0000 7f6ad78c2640 1 -- 172.21.15.160:0/4014402750 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad01aba40 con 0x7f6ad0104990 2024-09-16T08:45:47.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.262+0000 7f6ad78c2640 1 -- 172.21.15.160:0/4014402750 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f6ad01ac070 con 0x7f6ad0104990 2024-09-16T08:45:47.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.262+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f6ac0000f50 con 0x7f6ad0104990 2024-09-16T08:45:47.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.262+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f6ac0004c50 con 0x7f6ad0104990 2024-09-16T08:45:47.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.262+0000 7f6ad78c2640 1 -- 172.21.15.160:0/4014402750 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab40050b0 con 0x7f6ad0104990 2024-09-16T08:45:47.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.263+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 16) ==== 103096+0+0 (unknown 2118139471 0 0) 0x7f6ac0001b00 con 0x7f6ad0104990 2024-09-16T08:45:47.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.264+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(63..63 src has 1..63) ==== 5942+0+0 (unknown 3139999450 0 0) 0x7f6ac0098220 con 0x7f6ad0104990 2024-09-16T08:45:47.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.268+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f6ac005f410 con 0x7f6ad0104990 2024-09-16T08:45:47.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.513+0000 7f6ad78c2640 1 -- 172.21.15.160:0/4014402750 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f6ab40053a0 con 0x7f6ad0104990 2024-09-16T08:45:47.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.515+0000 7f6ac67fc640 1 -- 172.21.15.160:0/4014402750 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) ==== 170+0+59 (unknown 57899324 0 3076230736) 0x7f6ac00637f0 con 0x7f6ad0104990 2024-09-16T08:45:47.515 INFO:teuthology.orchestra.run.smithi160.stdout:[client.0] 2024-09-16T08:45:47.515 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQA78OdmtUKnHhAAYUS+KLUSNlhHpSr59sPiVg== 2024-09-16T08:45:47.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.517+0000 7f6ad78c2640 1 -- 172.21.15.160:0/4014402750 >> v1:172.21.15.160:6800/1286008384 conn(0x7f6abc07bc40 legacy=0x7f6abc07e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:47.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.517+0000 7f6ad78c2640 1 -- 172.21.15.160:0/4014402750 >> v1:172.21.15.160:6789/0 conn(0x7f6ad0104990 legacy=0x7f6ad01a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:47.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.518+0000 7f6ad78c2640 1 -- 172.21.15.160:0/4014402750 shutdown_connections 2024-09-16T08:45:47.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.518+0000 7f6ad78c2640 1 -- 172.21.15.160:0/4014402750 >> 172.21.15.160:0/4014402750 conn(0x7f6ad00fcf80 msgr2=0x7f6ad00ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:47.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.518+0000 7f6ad78c2640 1 -- 172.21.15.160:0/4014402750 shutdown_connections 2024-09-16T08:45:47.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:47.518+0000 7f6ad78c2640 1 -- 172.21.15.160:0/4014402750 wait complete. 2024-09-16T08:45:47.890 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:47 smithi160 ceph-mon[30137]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s rd, 5.7 KiB/s wr, 185 op/s 2024-09-16T08:45:47.890 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:47 smithi160 ceph-mon[30137]: mgrmap e16: y(active, since 5m), standbys: x 2024-09-16T08:45:47.890 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4014402750' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T08:45:47.890 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4014402750' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T08:45:47.890 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:47 smithi160 ceph-mon[40028]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s rd, 5.7 KiB/s wr, 185 op/s 2024-09-16T08:45:47.891 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:47 smithi160 ceph-mon[40028]: mgrmap e16: y(active, since 5m), standbys: x 2024-09-16T08:45:47.891 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4014402750' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T08:45:47.891 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4014402750' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T08:45:48.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:47 smithi169 ceph-mon[33465]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s rd, 5.7 KiB/s wr, 185 op/s 2024-09-16T08:45:48.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:47 smithi169 ceph-mon[33465]: mgrmap e16: y(active, since 5m), standbys: x 2024-09-16T08:45:48.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4014402750' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T08:45:48.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4014402750' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T08:45:48.317 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:45:48.317 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-16T08:45:48.317 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T08:45:48.355 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T08:45:49.019 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:45:49.650 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.598+0000 7f7c63b17640 1 -- 172.21.15.169:0/3938572568 >> v1:172.21.15.169:6789/0 conn(0x7f7c5c075c70 legacy=0x7f7c5c10b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:49.650 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.598+0000 7f7c63b17640 1 -- 172.21.15.169:0/3938572568 shutdown_connections 2024-09-16T08:45:49.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.598+0000 7f7c63b17640 1 -- 172.21.15.169:0/3938572568 >> 172.21.15.169:0/3938572568 conn(0x7f7c5c0fe060 msgr2=0x7f7c5c1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:49.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.598+0000 7f7c63b17640 1 -- 172.21.15.169:0/3938572568 shutdown_connections 2024-09-16T08:45:49.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.598+0000 7f7c63b17640 1 -- 172.21.15.169:0/3938572568 wait complete. 2024-09-16T08:45:49.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.598+0000 7f7c63b17640 1 Processor -- start 2024-09-16T08:45:49.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.598+0000 7f7c63b17640 1 -- start start 2024-09-16T08:45:49.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c63b17640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7c5c1a5130 con 0x7f7c5c075c70 2024-09-16T08:45:49.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c63b17640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7c5c1a6310 con 0x7f7c5c076f00 2024-09-16T08:45:49.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c63b17640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7c5c1a74f0 con 0x7f7c5c10ce70 2024-09-16T08:45:49.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c610b2640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f7c5c076f00 0x7f7c5c1a02e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:36000/0 (socket says 172.21.15.169:36000) 2024-09-16T08:45:49.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c610b2640 1 -- 172.21.15.169:0/2024909919 learned_addr learned my addr 172.21.15.169:0/2024909919 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-09-16T08:45:49.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1714871088 0 0) 0x7f7c5c1a6310 con 0x7f7c5c076f00 2024-09-16T08:45:49.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c48003520 con 0x7f7c5c076f00 2024-09-16T08:45:49.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1771385485 0 0) 0x7f7c5c1a74f0 con 0x7f7c5c10ce70 2024-09-16T08:45:49.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c5c1a6310 con 0x7f7c5c10ce70 2024-09-16T08:45:49.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1881361672 0 0) 0x7f7c5c1a5130 con 0x7f7c5c075c70 2024-09-16T08:45:49.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c5c1a74f0 con 0x7f7c5c075c70 2024-09-16T08:45:49.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1877837828 0 0) 0x7f7c48003520 con 0x7f7c5c076f00 2024-09-16T08:45:49.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7c5c1a5130 con 0x7f7c5c076f00 2024-09-16T08:45:49.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f7c4c001af0 con 0x7f7c5c076f00 2024-09-16T08:45:49.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4145123543 0 0) 0x7f7c5c1a6310 con 0x7f7c5c10ce70 2024-09-16T08:45:49.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7c48003520 con 0x7f7c5c10ce70 2024-09-16T08:45:49.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f7c54001b20 con 0x7f7c5c10ce70 2024-09-16T08:45:49.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2560005656 0 0) 0x7f7c5c1a74f0 con 0x7f7c5c075c70 2024-09-16T08:45:49.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7c5c1a6310 con 0x7f7c5c075c70 2024-09-16T08:45:49.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 <== mon.1 v1:172.21.15.169:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2218526862 0 0) 0x7f7c5c1a5130 con 0x7f7c5c076f00 2024-09-16T08:45:49.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 >> v1:172.21.15.160:6790/0 conn(0x7f7c5c10ce70 legacy=0x7f7c5c1a3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:49.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 >> v1:172.21.15.160:6789/0 conn(0x7f7c5c075c70 legacy=0x7f7c5c19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:49.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5c1a86d0 con 0x7f7c5c076f00 2024-09-16T08:45:49.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.599+0000 7f7c63b17640 1 -- 172.21.15.169:0/2024909919 --> v1:172.21.15.169:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5c1a5300 con 0x7f7c5c076f00 2024-09-16T08:45:49.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.600+0000 7f7c63b17640 1 -- 172.21.15.169:0/2024909919 --> v1:172.21.15.169:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7c5c1a5890 con 0x7f7c5c076f00 2024-09-16T08:45:49.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.600+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 <== mon.1 v1:172.21.15.169:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f7c4c001d20 con 0x7f7c5c076f00 2024-09-16T08:45:49.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.600+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 <== mon.1 v1:172.21.15.169:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f7c4c004d40 con 0x7f7c5c076f00 2024-09-16T08:45:49.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.600+0000 7f7c63b17640 1 -- 172.21.15.169:0/2024909919 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c400050b0 con 0x7f7c5c076f00 2024-09-16T08:45:49.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.601+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 <== mon.1 v1:172.21.15.169:6789/0 7 ==== mgrmap(e 16) ==== 103096+0+0 (unknown 2118139471 0 0) 0x7f7c4c003800 con 0x7f7c5c076f00 2024-09-16T08:45:49.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.602+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 <== mon.1 v1:172.21.15.169:6789/0 8 ==== osd_map(63..63 src has 1..63) ==== 5942+0+0 (unknown 3139999450 0 0) 0x7f7c4c0994b0 con 0x7f7c5c076f00 2024-09-16T08:45:49.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.606+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 <== mon.1 v1:172.21.15.169:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f7c4c0606a0 con 0x7f7c5c076f00 2024-09-16T08:45:49.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:49 smithi169 ceph-mon[33465]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 139 op/s 2024-09-16T08:45:49.858 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.857+0000 7f7c63b17640 1 -- 172.21.15.169:0/2024909919 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f7c400053a0 con 0x7f7c5c076f00 2024-09-16T08:45:49.865 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.864+0000 7f7c52ffd640 1 -- 172.21.15.169:0/2024909919 <== mon.1 v1:172.21.15.169:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v18) ==== 170+0+59 (unknown 3508017507 0 1248688760) 0x7f7c4c064a80 con 0x7f7c5c076f00 2024-09-16T08:45:49.865 INFO:teuthology.orchestra.run.smithi169.stdout:[client.1] 2024-09-16T08:45:49.865 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA98Odm8AM5MxAAKBjbqx6tkc6Rogul1hPexQ== 2024-09-16T08:45:49.867 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.866+0000 7f7c63b17640 1 -- 172.21.15.169:0/2024909919 >> v1:172.21.15.160:6800/1286008384 conn(0x7f7c4807bd60 legacy=0x7f7c4807e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:49.867 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.866+0000 7f7c63b17640 1 -- 172.21.15.169:0/2024909919 >> v1:172.21.15.169:6789/0 conn(0x7f7c5c076f00 legacy=0x7f7c5c1a02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:49.867 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.866+0000 7f7c63b17640 1 -- 172.21.15.169:0/2024909919 shutdown_connections 2024-09-16T08:45:49.867 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.866+0000 7f7c63b17640 1 -- 172.21.15.169:0/2024909919 >> 172.21.15.169:0/2024909919 conn(0x7f7c5c0fe060 msgr2=0x7f7c5c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:49.867 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.867+0000 7f7c63b17640 1 -- 172.21.15.169:0/2024909919 shutdown_connections 2024-09-16T08:45:49.867 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T08:45:49.867+0000 7f7c63b17640 1 -- 172.21.15.169:0/2024909919 wait complete. 2024-09-16T08:45:49.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:49 smithi160 ceph-mon[30137]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 139 op/s 2024-09-16T08:45:49.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:49 smithi160 ceph-mon[40028]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 139 op/s 2024-09-16T08:45:50.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.169:0/2024909919' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T08:45:50.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:50 smithi160 ceph-mon[30137]: from='client.24511 ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T08:45:50.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:50 smithi160 ceph-mon[30137]: from='client.24511 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T08:45:50.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.169:0/2024909919' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T08:45:50.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:50 smithi160 ceph-mon[40028]: from='client.24511 ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T08:45:50.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:50 smithi160 ceph-mon[40028]: from='client.24511 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T08:45:51.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.169:0/2024909919' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T08:45:51.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:50 smithi169 ceph-mon[33465]: from='client.24511 ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T08:45:51.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:50 smithi169 ceph-mon[33465]: from='client.24511 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T08:45:51.579 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T08:45:51.609 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-16T08:45:51.609 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T08:45:51.646 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph config log 1 --format=json 2024-09-16T08:45:51.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:51 smithi169 ceph-mon[33465]: pgmap v205: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s rd, 3.9 KiB/s wr, 125 op/s 2024-09-16T08:45:51.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:51 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:51.903 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:51.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:51 smithi160 ceph-mon[30137]: pgmap v205: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s rd, 3.9 KiB/s wr, 125 op/s 2024-09-16T08:45:51.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:51 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:51.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:51 smithi160 ceph-mon[40028]: pgmap v205: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s rd, 3.9 KiB/s wr, 125 op/s 2024-09-16T08:45:51.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:51 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:52.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.358+0000 7f7d57107640 1 -- 172.21.15.160:0/1264651699 >> v1:172.21.15.160:6789/0 conn(0x7f7d5010b690 legacy=0x7f7d5010da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:52.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.358+0000 7f7d57107640 1 -- 172.21.15.160:0/1264651699 shutdown_connections 2024-09-16T08:45:52.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.358+0000 7f7d57107640 1 -- 172.21.15.160:0/1264651699 >> 172.21.15.160:0/1264651699 conn(0x7f7d500fe5b0 msgr2=0x7f7d501009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:52.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.358+0000 7f7d57107640 1 -- 172.21.15.160:0/1264651699 shutdown_connections 2024-09-16T08:45:52.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.358+0000 7f7d57107640 1 -- 172.21.15.160:0/1264651699 wait complete. 2024-09-16T08:45:52.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.359+0000 7f7d57107640 1 Processor -- start 2024-09-16T08:45:52.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.359+0000 7f7d57107640 1 -- start start 2024-09-16T08:45:52.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.359+0000 7f7d57107640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7d501a99b0 con 0x7f7d5010b690 2024-09-16T08:45:52.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.359+0000 7f7d57107640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7d501aabb0 con 0x7f7d5010f230 2024-09-16T08:45:52.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.359+0000 7f7d57107640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7d501abdb0 con 0x7f7d50102b20 2024-09-16T08:45:52.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.359+0000 7f7d54ea3640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f7d50102b20 0x7f7d50076cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:35144/0 (socket says 172.21.15.160:35144) 2024-09-16T08:45:52.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.359+0000 7f7d54ea3640 1 -- 172.21.15.160:0/3622683116 learned_addr learned my addr 172.21.15.160:0/3622683116 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:45:52.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2517628426 0 0) 0x7f7d501abdb0 con 0x7f7d50102b20 2024-09-16T08:45:52.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d40003520 con 0x7f7d50102b20 2024-09-16T08:45:52.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3862479322 0 0) 0x7f7d501a99b0 con 0x7f7d5010b690 2024-09-16T08:45:52.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d501abdb0 con 0x7f7d5010b690 2024-09-16T08:45:52.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2037806242 0 0) 0x7f7d501aabb0 con 0x7f7d5010f230 2024-09-16T08:45:52.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d501a99b0 con 0x7f7d5010f230 2024-09-16T08:45:52.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3656930008 0 0) 0x7f7d40003520 con 0x7f7d50102b20 2024-09-16T08:45:52.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7d501aabb0 con 0x7f7d50102b20 2024-09-16T08:45:52.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f7d44001350 con 0x7f7d50102b20 2024-09-16T08:45:52.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4232867710 0 0) 0x7f7d501abdb0 con 0x7f7d5010b690 2024-09-16T08:45:52.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7d40003520 con 0x7f7d5010b690 2024-09-16T08:45:52.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f7d3c001650 con 0x7f7d5010b690 2024-09-16T08:45:52.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 500110136 0 0) 0x7f7d501a99b0 con 0x7f7d5010f230 2024-09-16T08:45:52.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7d501abdb0 con 0x7f7d5010f230 2024-09-16T08:45:52.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f7d48001b70 con 0x7f7d5010f230 2024-09-16T08:45:52.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 880417228 0 0) 0x7f7d501aabb0 con 0x7f7d50102b20 2024-09-16T08:45:52.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 >> v1:172.21.15.169:6789/0 conn(0x7f7d5010f230 legacy=0x7f7d501a81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:52.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 >> v1:172.21.15.160:6789/0 conn(0x7f7d5010b690 legacy=0x7f7d501a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:52.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d501ad050 con 0x7f7d50102b20 2024-09-16T08:45:52.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d57107640 1 -- 172.21.15.160:0/3622683116 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7d501abfe0 con 0x7f7d50102b20 2024-09-16T08:45:52.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f7d44001630 con 0x7f7d50102b20 2024-09-16T08:45:52.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f7d44005c50 con 0x7f7d50102b20 2024-09-16T08:45:52.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.360+0000 7f7d57107640 1 -- 172.21.15.160:0/3622683116 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f7d501ac590 con 0x7f7d50102b20 2024-09-16T08:45:52.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.361+0000 7f7d57107640 1 -- 172.21.15.160:0/3622683116 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d50103dd0 con 0x7f7d50102b20 2024-09-16T08:45:52.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.362+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 16) ==== 103096+0+0 (unknown 2118139471 0 0) 0x7f7d440038a0 con 0x7f7d50102b20 2024-09-16T08:45:52.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.363+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(63..63 src has 1..63) ==== 5942+0+0 (unknown 3139999450 0 0) 0x7f7d44099330 con 0x7f7d50102b20 2024-09-16T08:45:52.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.367+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f7d440605c0 con 0x7f7d50102b20 2024-09-16T08:45:52.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.543+0000 7f7d57107640 1 -- 172.21.15.160:0/3622683116 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f7d50114fe0 con 0x7f7d50102b20 2024-09-16T08:45:52.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.544+0000 7f7d4dffb640 1 -- 172.21.15.160:0/3622683116 <== mon.2 v1:172.21.15.160:6790/0 10 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v24) ==== 86+0+187 (unknown 940796443 0 4236826948) 0x7f7d440649a0 con 0x7f7d50102b20 2024-09-16T08:45:52.544 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:45:52.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.546+0000 7f7d57107640 1 -- 172.21.15.160:0/3622683116 >> v1:172.21.15.160:6800/1286008384 conn(0x7f7d4007bc30 legacy=0x7f7d4007e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:52.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.546+0000 7f7d57107640 1 -- 172.21.15.160:0/3622683116 >> v1:172.21.15.160:6790/0 conn(0x7f7d50102b20 legacy=0x7f7d50076cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:52.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.546+0000 7f7d57107640 1 -- 172.21.15.160:0/3622683116 shutdown_connections 2024-09-16T08:45:52.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.546+0000 7f7d57107640 1 -- 172.21.15.160:0/3622683116 >> 172.21.15.160:0/3622683116 conn(0x7f7d500fe5b0 msgr2=0x7f7d50101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:52.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.546+0000 7f7d57107640 1 -- 172.21.15.160:0/3622683116 shutdown_connections 2024-09-16T08:45:52.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:52.546+0000 7f7d57107640 1 -- 172.21.15.160:0/3622683116 wait complete. 2024-09-16T08:45:52.897 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3622683116' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-16T08:45:52.897 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3622683116' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-16T08:45:53.007 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3622683116' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-16T08:45:53.488 INFO:teuthology.orchestra.run.smithi160.stdout:[{"version":24,"timestamp":"2024-09-16T08:45:41.072630+0000","name":"","changes":[{"name":"mgr/mgr/dashboard/RGW_API_SECRET_KEY","new_value":"vCmL7nwJt81BtYzIxzZ8B3lkBZHEH3NAhteB8pFL"}]}] 2024-09-16T08:45:53.489 INFO:tasks.ceph_manager:config epoch is 24 2024-09-16T08:45:53.489 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T08:45:53.489 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-16T08:45:53.489 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-16T08:45:53.737 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:53.795 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:53 smithi169 ceph-mon[33465]: pgmap v206: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-09-16T08:45:53.858 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:53 smithi160 ceph-mon[30137]: pgmap v206: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-09-16T08:45:53.858 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:53 smithi160 ceph-mon[40028]: pgmap v206: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-09-16T08:45:54.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.213+0000 7fede9dc7640 1 -- 172.21.15.160:0/1896482441 >> v1:172.21.15.160:6790/0 conn(0x7fede4101a20 legacy=0x7fede4101e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:54.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.213+0000 7fede9dc7640 1 -- 172.21.15.160:0/1896482441 shutdown_connections 2024-09-16T08:45:54.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.213+0000 7fede9dc7640 1 -- 172.21.15.160:0/1896482441 >> 172.21.15.160:0/1896482441 conn(0x7fede40fd4b0 msgr2=0x7fede40ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:54.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.213+0000 7fede9dc7640 1 -- 172.21.15.160:0/1896482441 shutdown_connections 2024-09-16T08:45:54.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.213+0000 7fede9dc7640 1 -- 172.21.15.160:0/1896482441 wait complete. 2024-09-16T08:45:54.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.213+0000 7fede9dc7640 1 Processor -- start 2024-09-16T08:45:54.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.213+0000 7fede9dc7640 1 -- start start 2024-09-16T08:45:54.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede9dc7640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fede41a77b0 con 0x7fede410a590 2024-09-16T08:45:54.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede9dc7640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fede41a89b0 con 0x7fede4101a20 2024-09-16T08:45:54.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede9dc7640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fede41a9bb0 con 0x7fede410e130 2024-09-16T08:45:54.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7feddbfff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fede410a590 0x7fede41a2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:44628/0 (socket says 172.21.15.160:44628) 2024-09-16T08:45:54.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7feddbfff640 1 -- 172.21.15.160:0/3344425653 learned_addr learned my addr 172.21.15.160:0/3344425653 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:45:54.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede3fff640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7fede410e130 0x7fede41a5eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:35166/0 (socket says 172.21.15.160:35166) 2024-09-16T08:45:54.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3851248257 0 0) 0x7fede41a9bb0 con 0x7fede410e130 2024-09-16T08:45:54.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedd0003520 con 0x7fede410e130 2024-09-16T08:45:54.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1023952524 0 0) 0x7fede41a77b0 con 0x7fede410a590 2024-09-16T08:45:54.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fede41a9bb0 con 0x7fede410a590 2024-09-16T08:45:54.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1030744529 0 0) 0x7fede41a89b0 con 0x7fede4101a20 2024-09-16T08:45:54.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fede41a77b0 con 0x7fede4101a20 2024-09-16T08:45:54.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3602800909 0 0) 0x7fede41a9bb0 con 0x7fede410a590 2024-09-16T08:45:54.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fede41a89b0 con 0x7fede410a590 2024-09-16T08:45:54.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fedcc0013b0 con 0x7fede410a590 2024-09-16T08:45:54.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2283870993 0 0) 0x7fedd0003520 con 0x7fede410e130 2024-09-16T08:45:54.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fede41a9bb0 con 0x7fede410e130 2024-09-16T08:45:54.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3085583663 0 0) 0x7fede41a77b0 con 0x7fede4101a20 2024-09-16T08:45:54.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fedd0003520 con 0x7fede4101a20 2024-09-16T08:45:54.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7feddc001bf0 con 0x7fede410e130 2024-09-16T08:45:54.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fedd4001430 con 0x7fede4101a20 2024-09-16T08:45:54.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2353993291 0 0) 0x7fede41a89b0 con 0x7fede410a590 2024-09-16T08:45:54.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 >> v1:172.21.15.160:6790/0 conn(0x7fede410e130 legacy=0x7fede41a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:54.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 >> v1:172.21.15.169:6789/0 conn(0x7fede4101a20 legacy=0x7fede410d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:54.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede41aae50 con 0x7fede410a590 2024-09-16T08:45:54.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede9dc7640 1 -- 172.21.15.160:0/3344425653 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fede41a79e0 con 0x7fede410a590 2024-09-16T08:45:54.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.214+0000 7fede9dc7640 1 -- 172.21.15.160:0/3344425653 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fede41a8010 con 0x7fede410a590 2024-09-16T08:45:54.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.215+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fedcc003a60 con 0x7fede410a590 2024-09-16T08:45:54.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.215+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fedcc005be0 con 0x7fede410a590 2024-09-16T08:45:54.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.215+0000 7fede9dc7640 1 -- 172.21.15.160:0/3344425653 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc4005080 con 0x7fede410a590 2024-09-16T08:45:54.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.216+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 16) ==== 103096+0+0 (unknown 2118139471 0 0) 0x7fedcc0035f0 con 0x7fede410a590 2024-09-16T08:45:54.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.217+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(63..63 src has 1..63) ==== 5942+0+0 (unknown 3139999450 0 0) 0x7fedcc099070 con 0x7fede410a590 2024-09-16T08:45:54.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.221+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fedcc060220 con 0x7fede410a590 2024-09-16T08:45:54.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.443+0000 7fede9dc7640 1 -- 172.21.15.160:0/3344425653 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fedc4005370 con 0x7fede410a590 2024-09-16T08:45:54.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.446+0000 7fede17fa640 1 -- 172.21.15.160:0/3344425653 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) ==== 74+0+197735 (unknown 3479712753 0 2436045295) 0x7fedcc064600 con 0x7fede410a590 2024-09-16T08:45:54.449 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:45:54.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.450+0000 7fede9dc7640 1 -- 172.21.15.160:0/3344425653 >> v1:172.21.15.160:6800/1286008384 conn(0x7fedd007bbd0 legacy=0x7fedd007e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:54.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.450+0000 7fede9dc7640 1 -- 172.21.15.160:0/3344425653 >> v1:172.21.15.160:6789/0 conn(0x7fede410a590 legacy=0x7fede41a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:54.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.450+0000 7fede9dc7640 1 -- 172.21.15.160:0/3344425653 shutdown_connections 2024-09-16T08:45:54.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.450+0000 7fede9dc7640 1 -- 172.21.15.160:0/3344425653 >> 172.21.15.160:0/3344425653 conn(0x7fede40fd4b0 msgr2=0x7fede406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:54.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.450+0000 7fede9dc7640 1 -- 172.21.15.160:0/3344425653 shutdown_connections 2024-09-16T08:45:54.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:54.450+0000 7fede9dc7640 1 -- 172.21.15.160:0/3344425653 wait complete. 2024-09-16T08:45:55.036 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3344425653' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-16T08:45:55.037 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:45:54 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:45:55.163 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3344425653' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-16T08:45:55.164 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3344425653' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-16T08:45:55.193 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":16,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6800","nonce":1286008384}]},"active_addr":"172.21.15.160:6800/1286008384","active_change":"2024-09-16T08:40:45.804497+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24121,"name":"x","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.160:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":2502912438}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":333344116}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":3682644134}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":1905191124}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":3551212808}]}]} 2024-09-16T08:45:55.198 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-16T08:45:55.198 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T08:45:55.198 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T08:45:55.446 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:55.705 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:55 smithi169 systemd[1]: Starting Ceph prometheus.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T08:45:55.849 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:55 smithi160 ceph-mon[30137]: pgmap v207: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 31 KiB/s rd, 2.3 KiB/s wr, 75 op/s 2024-09-16T08:45:55.849 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:55 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:45:55.850 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:55 smithi160 ceph-mon[40028]: pgmap v207: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 31 KiB/s rd, 2.3 KiB/s wr, 75 op/s 2024-09-16T08:45:55.850 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:55 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:45:55.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.930+0000 7ff1849e0640 1 -- 172.21.15.160:0/816593848 >> v1:172.21.15.160:6789/0 conn(0x7ff18010f230 legacy=0x7ff1801116f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:55.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.930+0000 7ff1849e0640 1 -- 172.21.15.160:0/816593848 shutdown_connections 2024-09-16T08:45:55.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.930+0000 7ff1849e0640 1 -- 172.21.15.160:0/816593848 >> 172.21.15.160:0/816593848 conn(0x7ff1800fe5b0 msgr2=0x7ff1801009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:55.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.931+0000 7ff1849e0640 1 -- 172.21.15.160:0/816593848 shutdown_connections 2024-09-16T08:45:55.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.931+0000 7ff1849e0640 1 -- 172.21.15.160:0/816593848 wait complete. 2024-09-16T08:45:55.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.931+0000 7ff1849e0640 1 Processor -- start 2024-09-16T08:45:55.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.931+0000 7ff1849e0640 1 -- start start 2024-09-16T08:45:55.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.931+0000 7ff1849e0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff1801a7750 con 0x7ff18010f230 2024-09-16T08:45:55.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.931+0000 7ff1849e0640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff1801a8950 con 0x7ff18010b690 2024-09-16T08:45:55.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.931+0000 7ff1849e0640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff1801a9b50 con 0x7ff180102b20 2024-09-16T08:45:55.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff17e59c640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7ff180102b20 0x7ff18010e8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:35180/0 (socket says 172.21.15.160:35180) 2024-09-16T08:45:55.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff17e59c640 1 -- 172.21.15.160:0/1124520789 learned_addr learned my addr 172.21.15.160:0/1124520789 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:45:55.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1180358391 0 0) 0x7ff1801a7750 con 0x7ff18010f230 2024-09-16T08:45:55.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff16c004920 con 0x7ff18010f230 2024-09-16T08:45:55.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1468066874 0 0) 0x7ff1801a9b50 con 0x7ff180102b20 2024-09-16T08:45:55.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1801a7750 con 0x7ff180102b20 2024-09-16T08:45:55.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3010626427 0 0) 0x7ff1801a8950 con 0x7ff18010b690 2024-09-16T08:45:55.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1801a9b50 con 0x7ff18010b690 2024-09-16T08:45:55.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1873940788 0 0) 0x7ff16c004920 con 0x7ff18010f230 2024-09-16T08:45:55.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff1801a8950 con 0x7ff18010f230 2024-09-16T08:45:55.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ff178001880 con 0x7ff18010f230 2024-09-16T08:45:55.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 546026185 0 0) 0x7ff1801a7750 con 0x7ff180102b20 2024-09-16T08:45:55.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff16c004920 con 0x7ff180102b20 2024-09-16T08:45:55.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4272270158 0 0) 0x7ff1801a9b50 con 0x7ff18010b690 2024-09-16T08:45:55.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff1801a7750 con 0x7ff18010b690 2024-09-16T08:45:55.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ff170001970 con 0x7ff180102b20 2024-09-16T08:45:55.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ff168001210 con 0x7ff18010b690 2024-09-16T08:45:55.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1546490238 0 0) 0x7ff1801a8950 con 0x7ff18010f230 2024-09-16T08:45:55.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 >> v1:172.21.15.160:6790/0 conn(0x7ff180102b20 legacy=0x7ff18010e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:55.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 >> v1:172.21.15.169:6789/0 conn(0x7ff18010b690 legacy=0x7ff1801a2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:55.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1801aadf0 con 0x7ff18010f230 2024-09-16T08:45:55.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1849e0640 1 -- 172.21.15.160:0/1124520789 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff1801a9d80 con 0x7ff18010f230 2024-09-16T08:45:55.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1849e0640 1 -- 172.21.15.160:0/1124520789 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff1801aa310 con 0x7ff18010f230 2024-09-16T08:45:55.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7ff178001cd0 con 0x7ff18010f230 2024-09-16T08:45:55.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.932+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ff178004dc0 con 0x7ff18010f230 2024-09-16T08:45:55.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.933+0000 7ff1849e0640 1 -- 172.21.15.160:0/1124520789 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff160005080 con 0x7ff18010f230 2024-09-16T08:45:55.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.934+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 16) ==== 103096+0+0 (unknown 2118139471 0 0) 0x7ff178003970 con 0x7ff18010f230 2024-09-16T08:45:55.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.935+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(63..63 src has 1..63) ==== 5942+0+0 (unknown 3139999450 0 0) 0x7ff1780980b0 con 0x7ff18010f230 2024-09-16T08:45:55.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:55.939+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7ff17805f320 con 0x7ff18010f230 2024-09-16T08:45:56.058 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:55 smithi169 ceph-mon[33465]: pgmap v207: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 31 KiB/s rd, 2.3 KiB/s wr, 75 op/s 2024-09-16T08:45:56.058 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:55 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:45:56.058 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:55 smithi169 podman[61877]: 2024-09-16 08:45:55.716599943 +0000 UTC m=+0.017410045 image pull 1d3b7f56885b6dd623f1785be963aa9c195f86bc256ea454e8d02a7980b79c53 quay.io/prometheus/prometheus:v2.51.0 2024-09-16T08:45:56.058 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:55 smithi169 podman[61877]: 2024-09-16 08:45:55.814268451 +0000 UTC m=+0.115078540 container create af370486808d78813b8ae3b6c4654f8e72eb181dd581d16b299312cde6c7866a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:45:56.058 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 podman[61877]: 2024-09-16 08:45:56.058194634 +0000 UTC m=+0.359004723 container init af370486808d78813b8ae3b6c4654f8e72eb181dd581d16b299312cde6c7866a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:45:56.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:56.118+0000 7ff1849e0640 1 -- 172.21.15.160:0/1124520789 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff160005370 con 0x7ff18010f230 2024-09-16T08:45:56.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:56.119+0000 7ff1777fe640 1 -- 172.21.15.160:0/1124520789 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+20753 (unknown 2546300363 0 2688136597) 0x7ff178063700 con 0x7ff18010f230 2024-09-16T08:45:56.120 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:45:56.120 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":63,"fsid":"27cca2fc-7407-11ef-bceb-c7b262605968","created":"2024-09-16T08:39:54.060780+0000","modified":"2024-09-16T08:45:44.068360+0000","last_up_change":"2024-09-16T08:45:23.119335+0000","last_in_change":"2024-09-16T08:45:00.473379+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T08:43:02.103619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-16T08:45:31.331921+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-16T08:45:32.039906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-16T08:45:33.880433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-16T08:45:35.843632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-16T08:45:37.910995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e77e8c1-25c0-4842-90a7-48769d62dbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6801","nonce":520428698}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6802","nonce":520428698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6804","nonce":520428698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6803","nonce":520428698}]},"public_addr":"172.21.15.160:6801/520428698","cluster_addr":"172.21.15.160:6802/520428698","heartbeat_back_addr":"172.21.15.160:6804/520428698","heartbeat_front_addr":"172.21.15.160:6803/520428698","state":["exists","up"]},{"osd":1,"uuid":"08f4dc4d-2873-45f3-a1d9-7c12f6e2c995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6805","nonce":704046158}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6806","nonce":704046158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6808","nonce":704046158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6807","nonce":704046158}]},"public_addr":"172.21.15.160:6805/704046158","cluster_addr":"172.21.15.160:6806/704046158","heartbeat_back_addr":"172.21.15.160:6808/704046158","heartbeat_front_addr":"172.21.15.160:6807/704046158","state":["exists","up"]},{"osd":2,"uuid":"0ba6c8a1-5676-4e1d-994a-a44a9b7d1996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6809","nonce":2138225188}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6810","nonce":2138225188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6812","nonce":2138225188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6811","nonce":2138225188}]},"public_addr":"172.21.15.160:6809/2138225188","cluster_addr":"172.21.15.160:6810/2138225188","heartbeat_back_addr":"172.21.15.160:6812/2138225188","heartbeat_front_addr":"172.21.15.160:6811/2138225188","state":["exists","up"]},{"osd":3,"uuid":"01400bbe-6dfe-4789-8b07-5bb3177eddfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6813","nonce":1752204566}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6814","nonce":1752204566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6816","nonce":1752204566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6815","nonce":1752204566}]},"public_addr":"172.21.15.160:6813/1752204566","cluster_addr":"172.21.15.160:6814/1752204566","heartbeat_back_addr":"172.21.15.160:6816/1752204566","heartbeat_front_addr":"172.21.15.160:6815/1752204566","state":["exists","up"]},{"osd":4,"uuid":"f0b51ca8-0851-4938-a811-667d57dbc18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6800","nonce":1893879009}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6801","nonce":1893879009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6803","nonce":1893879009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6802","nonce":1893879009}]},"public_addr":"172.21.15.169:6800/1893879009","cluster_addr":"172.21.15.169:6801/1893879009","heartbeat_back_addr":"172.21.15.169:6803/1893879009","heartbeat_front_addr":"172.21.15.169:6802/1893879009","state":["exists","up"]},{"osd":5,"uuid":"15a1f6ce-7c1b-4ebd-ab8a-4bacdbe5df55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6804","nonce":4217518758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6805","nonce":4217518758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6807","nonce":4217518758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6806","nonce":4217518758}]},"public_addr":"172.21.15.169:6804/4217518758","cluster_addr":"172.21.15.169:6805/4217518758","heartbeat_back_addr":"172.21.15.169:6807/4217518758","heartbeat_front_addr":"172.21.15.169:6806/4217518758","state":["exists","up"]},{"osd":6,"uuid":"c3a03867-0d98-41bb-bbd7-cd6fb4f67eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6808","nonce":3462109725}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6809","nonce":3462109725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6811","nonce":3462109725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6810","nonce":3462109725}]},"public_addr":"172.21.15.169:6808/3462109725","cluster_addr":"172.21.15.169:6809/3462109725","heartbeat_back_addr":"172.21.15.169:6811/3462109725","heartbeat_front_addr":"172.21.15.169:6810/3462109725","state":["exists","up"]},{"osd":7,"uuid":"a24f20d7-85f2-4c15-bab9-05382f876001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6812","nonce":1196718305}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6813","nonce":1196718305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6815","nonce":1196718305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6814","nonce":1196718305}]},"public_addr":"172.21.15.169:6812/1196718305","cluster_addr":"172.21.15.169:6813/1196718305","heartbeat_back_addr":"172.21.15.169:6815/1196718305","heartbeat_front_addr":"172.21.15.169:6814/1196718305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:42:02.669441+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:42:30.592545+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:42:58.969509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:43:26.692920+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:43:53.561518+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:44:22.355115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:44:51.383158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:45:21.104100+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/797617578":"2024-09-17T08:40:45.804394+0000","172.21.15.160:0/2198066550":"2024-09-17T08:40:45.804394+0000","172.21.15.160:0/3719822124":"2024-09-17T08:40:45.804394+0000","172.21.15.160:6800/3425583652":"2024-09-17T08:40:45.804394+0000","172.21.15.160:0/979267250":"2024-09-17T08:40:20.860608+0000","172.21.15.160:0/17466509":"2024-09-17T08:40:20.860608+0000","172.21.15.160:6800/2809527782":"2024-09-17T08:40:20.860608+0000","172.21.15.160:0/412999819":"2024-09-17T08:40:20.860608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T08:45:56.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:56.121+0000 7ff1849e0640 1 -- 172.21.15.160:0/1124520789 >> v1:172.21.15.160:6800/1286008384 conn(0x7ff16c07bd50 legacy=0x7ff16c07e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:56.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:56.121+0000 7ff1849e0640 1 -- 172.21.15.160:0/1124520789 >> v1:172.21.15.160:6789/0 conn(0x7ff18010f230 legacy=0x7ff1801a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:56.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:56.122+0000 7ff1849e0640 1 -- 172.21.15.160:0/1124520789 shutdown_connections 2024-09-16T08:45:56.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:56.122+0000 7ff1849e0640 1 -- 172.21.15.160:0/1124520789 >> 172.21.15.160:0/1124520789 conn(0x7ff1800fe5b0 msgr2=0x7ff180106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:56.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:56.122+0000 7ff1849e0640 1 -- 172.21.15.160:0/1124520789 shutdown_connections 2024-09-16T08:45:56.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:56.122+0000 7ff1849e0640 1 -- 172.21.15.160:0/1124520789 wait complete. 2024-09-16T08:45:56.350 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 podman[61877]: 2024-09-16 08:45:56.063359518 +0000 UTC m=+0.364169608 container start af370486808d78813b8ae3b6c4654f8e72eb181dd581d16b299312cde6c7866a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:45:56.350 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.082Z caller=main.go:617 level=info msg="Starting Prometheus Server" mode=server version="(version=2.51.0, branch=HEAD, revision=c05c15512acb675e3f6cd662a6727854e93fc024)" 2024-09-16T08:45:56.351 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.082Z caller=main.go:622 level=info build_context="(go=go1.22.1, platform=linux/amd64, user=root@b5723e458358, date=20240319-10:54:45, tags=netgo,builtinassets,stringlabels)" 2024-09-16T08:45:56.351 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.082Z caller=main.go:623 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi169 (none))" 2024-09-16T08:45:56.351 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.082Z caller=main.go:624 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-16T08:45:56.351 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.082Z caller=main.go:625 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-16T08:45:56.351 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.083Z caller=web.go:568 level=info component=web msg="Start listening for connections" address=:9095 2024-09-16T08:45:56.351 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.084Z caller=main.go:1129 level=info msg="Starting TSDB ..." 2024-09-16T08:45:56.352 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.086Z caller=tls_config.go:313 level=info component=web msg="Listening on" address=[::]:9095 2024-09-16T08:45:56.352 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.086Z caller=tls_config.go:316 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-16T08:45:56.352 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.088Z caller=head.go:616 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-16T08:45:56.352 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.088Z caller=head.go:698 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.713µs 2024-09-16T08:45:56.352 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.088Z caller=head.go:706 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-16T08:45:56.352 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.088Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-16T08:45:56.353 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.088Z caller=head.go:815 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=23.758µs wal_replay_duration=162.461µs wbl_replay_duration=94ns total_replay_duration=204.348µs 2024-09-16T08:45:56.353 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.089Z caller=main.go:1150 level=info fs_type=XFS_SUPER_MAGIC 2024-09-16T08:45:56.353 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.089Z caller=main.go:1153 level=info msg="TSDB started" 2024-09-16T08:45:56.353 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.090Z caller=main.go:1335 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-16T08:45:56.353 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.105Z caller=main.go:1372 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.576842ms db_storage=773ns remote_storage=1.103µs web_handler=342ns query_engine=585ns scrape=499.674µs scrape_sd=135.403µs notify=925ns notify_sd=861ns rules=14.534967ms tracing=5.566µs 2024-09-16T08:45:56.354 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.105Z caller=main.go:1114 level=info msg="Server is ready to receive web requests." 2024-09-16T08:45:56.354 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:45:56.105Z caller=manager.go:163 level=info component="rule manager" msg="Starting rule manager..." 2024-09-16T08:45:56.706 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 bash[61877]: af370486808d78813b8ae3b6c4654f8e72eb181dd581d16b299312cde6c7866a 2024-09-16T08:45:56.706 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:45:56 smithi169 systemd[1]: Started Ceph prometheus.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:45:56.866 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T08:45:56.866 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T08:45:57.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1124520789' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T08:45:57.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:57.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:57.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:57.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:56 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-16T08:45:57.111 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:57.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1124520789' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T08:45:57.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:57.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:57.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:57.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:56 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-16T08:45:57.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1124520789' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T08:45:57.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:57.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:57.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' 2024-09-16T08:45:57.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:56 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-16T08:45:57.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.620+0000 7fe80394e640 1 -- 172.21.15.160:0/837970082 >> v1:172.21.15.160:6790/0 conn(0x7fe7fc101a00 legacy=0x7fe7fc101de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:57.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.620+0000 7fe80394e640 1 -- 172.21.15.160:0/837970082 shutdown_connections 2024-09-16T08:45:57.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.620+0000 7fe80394e640 1 -- 172.21.15.160:0/837970082 >> 172.21.15.160:0/837970082 conn(0x7fe7fc0fd450 msgr2=0x7fe7fc0ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:57.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.620+0000 7fe80394e640 1 -- 172.21.15.160:0/837970082 shutdown_connections 2024-09-16T08:45:57.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.620+0000 7fe80394e640 1 -- 172.21.15.160:0/837970082 wait complete. 2024-09-16T08:45:57.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.621+0000 7fe80394e640 1 Processor -- start 2024-09-16T08:45:57.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.621+0000 7fe80394e640 1 -- start start 2024-09-16T08:45:57.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.621+0000 7fe80394e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe7fc1a7770 con 0x7fe7fc10a570 2024-09-16T08:45:57.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.621+0000 7fe80394e640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe7fc1a8970 con 0x7fe7fc10e110 2024-09-16T08:45:57.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.621+0000 7fe80394e640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe7fc1a9b70 con 0x7fe7fc101a00 2024-09-16T08:45:57.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.621+0000 7fe8016ea640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7fe7fc101a00 0x7fe7fc10d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:50882/0 (socket says 172.21.15.160:50882) 2024-09-16T08:45:57.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.621+0000 7fe8016ea640 1 -- 172.21.15.160:0/911790395 learned_addr learned my addr 172.21.15.160:0/911790395 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:45:57.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1051067109 0 0) 0x7fe7fc1a7770 con 0x7fe7fc10a570 2024-09-16T08:45:57.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7e8003160 con 0x7fe7fc10a570 2024-09-16T08:45:57.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3443088054 0 0) 0x7fe7fc1a9b70 con 0x7fe7fc101a00 2024-09-16T08:45:57.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7fc1a7770 con 0x7fe7fc101a00 2024-09-16T08:45:57.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3934130797 0 0) 0x7fe7fc1a8970 con 0x7fe7fc10e110 2024-09-16T08:45:57.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7fc1a9b70 con 0x7fe7fc10e110 2024-09-16T08:45:57.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1945522839 0 0) 0x7fe7fc1a7770 con 0x7fe7fc101a00 2024-09-16T08:45:57.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe7fc1a8970 con 0x7fe7fc101a00 2024-09-16T08:45:57.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fe7f8001ad0 con 0x7fe7fc101a00 2024-09-16T08:45:57.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1625828556 0 0) 0x7fe7e8003160 con 0x7fe7fc10a570 2024-09-16T08:45:57.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe7fc1a7770 con 0x7fe7fc10a570 2024-09-16T08:45:57.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fe7ec001870 con 0x7fe7fc10a570 2024-09-16T08:45:57.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3489377450 0 0) 0x7fe7fc1a9b70 con 0x7fe7fc10e110 2024-09-16T08:45:57.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe7e8003160 con 0x7fe7fc10e110 2024-09-16T08:45:57.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fe7f40014c0 con 0x7fe7fc10e110 2024-09-16T08:45:57.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1913794909 0 0) 0x7fe7fc1a8970 con 0x7fe7fc101a00 2024-09-16T08:45:57.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 >> v1:172.21.15.169:6789/0 conn(0x7fe7fc10e110 legacy=0x7fe7fc1a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:57.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 >> v1:172.21.15.160:6789/0 conn(0x7fe7fc10a570 legacy=0x7fe7fc1a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:57.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7fc1aae10 con 0x7fe7fc101a00 2024-09-16T08:45:57.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe80394e640 1 -- 172.21.15.160:0/911790395 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe7fc1a9da0 con 0x7fe7fc101a00 2024-09-16T08:45:57.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.622+0000 7fe80394e640 1 -- 172.21.15.160:0/911790395 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fe7fc1aa3d0 con 0x7fe7fc101a00 2024-09-16T08:45:57.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.623+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fe7f8001d00 con 0x7fe7fc101a00 2024-09-16T08:45:57.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.623+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fe7f8004c30 con 0x7fe7fc101a00 2024-09-16T08:45:57.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.623+0000 7fe80394e640 1 -- 172.21.15.160:0/911790395 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e00050b0 con 0x7fe7fc101a00 2024-09-16T08:45:57.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.625+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 17) ==== 103110+0+0 (unknown 251840139 0 0) 0x7fe7f8004eb0 con 0x7fe7fc101a00 2024-09-16T08:45:57.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.625+0000 7fe800ee9640 1 -- 172.21.15.160:0/911790395 >> v1:172.21.15.160:6800/1286008384 conn(0x7fe7e807bb90 legacy=0x7fe7e807e050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/1286008384 2024-09-16T08:45:57.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.626+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(63..63 src has 1..63) ==== 5942+0+0 (unknown 3139999450 0 0) 0x7fe7f80991f0 con 0x7fe7fc101a00 2024-09-16T08:45:57.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.630+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fe7f8060480 con 0x7fe7fc101a00 2024-09-16T08:45:57.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.817+0000 7fe80394e640 1 -- 172.21.15.160:0/911790395 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7e00053a0 con 0x7fe7fc101a00 2024-09-16T08:45:57.818 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:57 smithi160 ceph-mon[30137]: pgmap v208: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 1.4 KiB/s wr, 45 op/s 2024-09-16T08:45:57.818 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:57 smithi160 ceph-mon[30137]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T08:45:57.818 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:57 smithi160 ceph-mon[30137]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-16T08:45:57.818 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:57 smithi160 ceph-mon[40028]: pgmap v208: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 1.4 KiB/s wr, 45 op/s 2024-09-16T08:45:57.819 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:57 smithi160 ceph-mon[40028]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T08:45:57.819 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:57 smithi160 ceph-mon[40028]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-16T08:45:57.819 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:45:57 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ignoring --setuser ceph since I am not root 2024-09-16T08:45:57.819 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:45:57 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ignoring --setgroup ceph since I am not root 2024-09-16T08:45:57.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.818+0000 7fe7f27fc640 1 -- 172.21.15.160:0/911790395 <== mon.2 v1:172.21.15.160:6790/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+20753 (unknown 2546300363 0 2688136597) 0x7fe7f8004c10 con 0x7fe7fc101a00 2024-09-16T08:45:57.820 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:45:57.821 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":63,"fsid":"27cca2fc-7407-11ef-bceb-c7b262605968","created":"2024-09-16T08:39:54.060780+0000","modified":"2024-09-16T08:45:44.068360+0000","last_up_change":"2024-09-16T08:45:23.119335+0000","last_in_change":"2024-09-16T08:45:00.473379+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T08:43:02.103619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-16T08:45:31.331921+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-16T08:45:32.039906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-16T08:45:33.880433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-16T08:45:35.843632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-16T08:45:37.910995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e77e8c1-25c0-4842-90a7-48769d62dbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6801","nonce":520428698}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6802","nonce":520428698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6804","nonce":520428698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6803","nonce":520428698}]},"public_addr":"172.21.15.160:6801/520428698","cluster_addr":"172.21.15.160:6802/520428698","heartbeat_back_addr":"172.21.15.160:6804/520428698","heartbeat_front_addr":"172.21.15.160:6803/520428698","state":["exists","up"]},{"osd":1,"uuid":"08f4dc4d-2873-45f3-a1d9-7c12f6e2c995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6805","nonce":704046158}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6806","nonce":704046158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6808","nonce":704046158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6807","nonce":704046158}]},"public_addr":"172.21.15.160:6805/704046158","cluster_addr":"172.21.15.160:6806/704046158","heartbeat_back_addr":"172.21.15.160:6808/704046158","heartbeat_front_addr":"172.21.15.160:6807/704046158","state":["exists","up"]},{"osd":2,"uuid":"0ba6c8a1-5676-4e1d-994a-a44a9b7d1996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6809","nonce":2138225188}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6810","nonce":2138225188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6812","nonce":2138225188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6811","nonce":2138225188}]},"public_addr":"172.21.15.160:6809/2138225188","cluster_addr":"172.21.15.160:6810/2138225188","heartbeat_back_addr":"172.21.15.160:6812/2138225188","heartbeat_front_addr":"172.21.15.160:6811/2138225188","state":["exists","up"]},{"osd":3,"uuid":"01400bbe-6dfe-4789-8b07-5bb3177eddfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6813","nonce":1752204566}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6814","nonce":1752204566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6816","nonce":1752204566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.160:6815","nonce":1752204566}]},"public_addr":"172.21.15.160:6813/1752204566","cluster_addr":"172.21.15.160:6814/1752204566","heartbeat_back_addr":"172.21.15.160:6816/1752204566","heartbeat_front_addr":"172.21.15.160:6815/1752204566","state":["exists","up"]},{"osd":4,"uuid":"f0b51ca8-0851-4938-a811-667d57dbc18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6800","nonce":1893879009}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6801","nonce":1893879009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6803","nonce":1893879009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6802","nonce":1893879009}]},"public_addr":"172.21.15.169:6800/1893879009","cluster_addr":"172.21.15.169:6801/1893879009","heartbeat_back_addr":"172.21.15.169:6803/1893879009","heartbeat_front_addr":"172.21.15.169:6802/1893879009","state":["exists","up"]},{"osd":5,"uuid":"15a1f6ce-7c1b-4ebd-ab8a-4bacdbe5df55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6804","nonce":4217518758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6805","nonce":4217518758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6807","nonce":4217518758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6806","nonce":4217518758}]},"public_addr":"172.21.15.169:6804/4217518758","cluster_addr":"172.21.15.169:6805/4217518758","heartbeat_back_addr":"172.21.15.169:6807/4217518758","heartbeat_front_addr":"172.21.15.169:6806/4217518758","state":["exists","up"]},{"osd":6,"uuid":"c3a03867-0d98-41bb-bbd7-cd6fb4f67eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6808","nonce":3462109725}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6809","nonce":3462109725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6811","nonce":3462109725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6810","nonce":3462109725}]},"public_addr":"172.21.15.169:6808/3462109725","cluster_addr":"172.21.15.169:6809/3462109725","heartbeat_back_addr":"172.21.15.169:6811/3462109725","heartbeat_front_addr":"172.21.15.169:6810/3462109725","state":["exists","up"]},{"osd":7,"uuid":"a24f20d7-85f2-4c15-bab9-05382f876001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6812","nonce":1196718305}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6813","nonce":1196718305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6815","nonce":1196718305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.169:6814","nonce":1196718305}]},"public_addr":"172.21.15.169:6812/1196718305","cluster_addr":"172.21.15.169:6813/1196718305","heartbeat_back_addr":"172.21.15.169:6815/1196718305","heartbeat_front_addr":"172.21.15.169:6814/1196718305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:42:02.669441+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:42:30.592545+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:42:58.969509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:43:26.692920+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:43:53.561518+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:44:22.355115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:44:51.383158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:45:21.104100+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/797617578":"2024-09-17T08:40:45.804394+0000","172.21.15.160:0/2198066550":"2024-09-17T08:40:45.804394+0000","172.21.15.160:0/3719822124":"2024-09-17T08:40:45.804394+0000","172.21.15.160:6800/3425583652":"2024-09-17T08:40:45.804394+0000","172.21.15.160:0/979267250":"2024-09-17T08:40:20.860608+0000","172.21.15.160:0/17466509":"2024-09-17T08:40:20.860608+0000","172.21.15.160:6800/2809527782":"2024-09-17T08:40:20.860608+0000","172.21.15.160:0/412999819":"2024-09-17T08:40:20.860608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T08:45:57.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.821+0000 7fe80394e640 1 -- 172.21.15.160:0/911790395 >> v1:172.21.15.160:6800/1286008384 conn(0x7fe7e807bb90 legacy=0x7fe7e807e050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T08:45:57.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.821+0000 7fe80394e640 1 -- 172.21.15.160:0/911790395 >> v1:172.21.15.160:6790/0 conn(0x7fe7fc101a00 legacy=0x7fe7fc10d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:45:57.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.821+0000 7fe80394e640 1 -- 172.21.15.160:0/911790395 shutdown_connections 2024-09-16T08:45:57.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.821+0000 7fe80394e640 1 -- 172.21.15.160:0/911790395 >> 172.21.15.160:0/911790395 conn(0x7fe7fc0fd450 msgr2=0x7fe7fc0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:45:57.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.821+0000 7fe80394e640 1 -- 172.21.15.160:0/911790395 shutdown_connections 2024-09-16T08:45:57.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:45:57.821+0000 7fe80394e640 1 -- 172.21.15.160:0/911790395 wait complete. 2024-09-16T08:45:57.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:57 smithi169 ceph-mon[33465]: pgmap v208: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 1.4 KiB/s wr, 45 op/s 2024-09-16T08:45:57.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:57 smithi169 ceph-mon[33465]: from='mgr.14150 v1:172.21.15.160:0/273788063' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T08:45:57.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:57 smithi169 ceph-mon[33465]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-16T08:45:57.850 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 08:45:57 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-x[35126]: ignoring --setuser ceph since I am not root 2024-09-16T08:45:57.850 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 08:45:57 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-x[35126]: ignoring --setgroup ceph since I am not root 2024-09-16T08:45:58.526 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T08:45:58.527 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T08:45:58.527 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T08:45:58.528 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T08:45:58.528 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T08:45:58.528 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T08:45:58.529 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-16T08:45:58.529 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-16T08:45:58.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:45:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/911790395' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T08:45:58.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:45:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/911790395' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T08:45:59.082 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:59.082 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:59.083 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:59.087 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:59.088 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:59.094 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:59.094 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:59.095 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:45:59.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:45:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/911790395' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T08:46:00.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.738+0000 7f9f5dc5e640 1 -- 172.21.15.160:0/3900161558 >> v1:172.21.15.160:6789/0 conn(0x7f9f58075540 legacy=0x7f9f58075920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:00.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.738+0000 7f9f5dc5e640 1 -- 172.21.15.160:0/3900161558 shutdown_connections 2024-09-16T08:46:00.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.738+0000 7f9f5dc5e640 1 -- 172.21.15.160:0/3900161558 >> 172.21.15.160:0/3900161558 conn(0x7f9f580fe5c0 msgr2=0x7f9f581009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:00.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.738+0000 7f9f5dc5e640 1 -- 172.21.15.160:0/3900161558 shutdown_connections 2024-09-16T08:46:00.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.738+0000 7f9f5dc5e640 1 -- 172.21.15.160:0/3900161558 wait complete. 2024-09-16T08:46:00.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.739+0000 7f9f5dc5e640 1 Processor -- start 2024-09-16T08:46:00.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.739+0000 7f9f5dc5e640 1 -- start start 2024-09-16T08:46:00.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.739+0000 7f9f5dc5e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9f581a9a10 con 0x7f9f58075540 2024-09-16T08:46:00.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.739+0000 7f9f5dc5e640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9f581aac10 con 0x7f9f58111460 2024-09-16T08:46:00.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.739+0000 7f9f5dc5e640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9f581abe10 con 0x7f9f580760f0 2024-09-16T08:46:00.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.739+0000 7f9f56ffd640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f9f580760f0 0x7f9f581a48b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:50922/0 (socket says 172.21.15.160:50922) 2024-09-16T08:46:00.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.739+0000 7f9f577fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f9f58075540 0x7f9f5810f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:43346/0 (socket says 172.21.15.160:43346) 2024-09-16T08:46:00.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.739+0000 7f9f56ffd640 1 -- 172.21.15.160:0/3388748321 learned_addr learned my addr 172.21.15.160:0/3388748321 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:00.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1301779413 0 0) 0x7f9f581a9a10 con 0x7f9f58075540 2024-09-16T08:46:00.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f40004920 con 0x7f9f58075540 2024-09-16T08:46:00.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2986775054 0 0) 0x7f9f581aac10 con 0x7f9f58111460 2024-09-16T08:46:00.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f581a9a10 con 0x7f9f58111460 2024-09-16T08:46:00.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 777600891 0 0) 0x7f9f581abe10 con 0x7f9f580760f0 2024-09-16T08:46:00.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f581aac10 con 0x7f9f580760f0 2024-09-16T08:46:00.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1993039476 0 0) 0x7f9f40004920 con 0x7f9f58075540 2024-09-16T08:46:00.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9f581abe10 con 0x7f9f58075540 2024-09-16T08:46:00.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f9f48001650 con 0x7f9f58075540 2024-09-16T08:46:00.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2390075458 0 0) 0x7f9f581a9a10 con 0x7f9f58111460 2024-09-16T08:46:00.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9f40004920 con 0x7f9f58111460 2024-09-16T08:46:00.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f9f50001bf0 con 0x7f9f58111460 2024-09-16T08:46:00.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1769296150 0 0) 0x7f9f581aac10 con 0x7f9f580760f0 2024-09-16T08:46:00.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9f581a9a10 con 0x7f9f580760f0 2024-09-16T08:46:00.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f9f4c001200 con 0x7f9f580760f0 2024-09-16T08:46:00.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2842100738 0 0) 0x7f9f581abe10 con 0x7f9f58075540 2024-09-16T08:46:00.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 >> v1:172.21.15.160:6790/0 conn(0x7f9f580760f0 legacy=0x7f9f581a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:00.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 >> v1:172.21.15.169:6789/0 conn(0x7f9f58111460 legacy=0x7f9f581a8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:00.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f581ad0b0 con 0x7f9f58075540 2024-09-16T08:46:00.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f5dc5e640 1 -- 172.21.15.160:0/3388748321 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9f581aae40 con 0x7f9f58075540 2024-09-16T08:46:00.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.740+0000 7f9f5dc5e640 1 -- 172.21.15.160:0/3388748321 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9f581ab3f0 con 0x7f9f58075540 2024-09-16T08:46:00.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.741+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f9f48001aa0 con 0x7f9f58075540 2024-09-16T08:46:00.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.741+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f9f48004b40 con 0x7f9f58075540 2024-09-16T08:46:00.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.742+0000 7f9f5dc5e640 1 -- 172.21.15.160:0/3388748321 --> v1:172.21.15.160:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f9f44000fb0 con 0x7f9f58075540 2024-09-16T08:46:00.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.742+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 17) ==== 103110+0+0 (unknown 251840139 0 0) 0x7f9f48004de0 con 0x7f9f58075540 2024-09-16T08:46:00.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.743+0000 7f9f56ffd640 1 -- 172.21.15.160:0/3388748321 >> v1:172.21.15.160:6800/1286008384 conn(0x7f9f4007bd80 legacy=0x7f9f4007e240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/1286008384 2024-09-16T08:46:00.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.743+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(63..63 src has 1..63) ==== 5942+0+0 (unknown 3139999450 0 0) 0x7f9f480991e0 con 0x7f9f58075540 2024-09-16T08:46:00.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.743+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 --> v1:172.21.15.160:6805/704046158 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f48005660 con 0x7f9f40086320 2024-09-16T08:46:00.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.743+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (unknown 2499814743 0 0) 0x7f9f48099630 con 0x7f9f58075540 2024-09-16T08:46:00.749 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.744+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== osd.1 v1:172.21.15.160:6805/704046158 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f9f48005660 con 0x7f9f40086320 2024-09-16T08:46:00.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.758+0000 7f9f5dc5e640 1 -- 172.21.15.160:0/3388748321 --> v1:172.21.15.160:6805/704046158 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9f44002ca0 con 0x7f9f40086320 2024-09-16T08:46:00.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.758+0000 7f9f54ff9640 1 -- 172.21.15.160:0/3388748321 <== osd.1 v1:172.21.15.160:6805/704046158 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 3910016750) 0x7f9f44002ca0 con 0x7f9f40086320 2024-09-16T08:46:00.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.759+0000 7f9f5dc5e640 1 -- 172.21.15.160:0/3388748321 >> v1:172.21.15.160:6805/704046158 conn(0x7f9f40086320 legacy=0x7f9f40088780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.759+0000 7f9f5dc5e640 1 -- 172.21.15.160:0/3388748321 >> v1:172.21.15.160:6800/1286008384 conn(0x7f9f4007bd80 legacy=0x7f9f4007e240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T08:46:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.759+0000 7f9f5dc5e640 1 -- 172.21.15.160:0/3388748321 >> v1:172.21.15.160:6789/0 conn(0x7f9f58075540 legacy=0x7f9f5810f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.759+0000 7f9f57fff640 1 -- 172.21.15.160:0/3388748321 reap_dead start 2024-09-16T08:46:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.759+0000 7f9f5dc5e640 1 -- 172.21.15.160:0/3388748321 shutdown_connections 2024-09-16T08:46:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.759+0000 7f9f5dc5e640 1 -- 172.21.15.160:0/3388748321 >> 172.21.15.160:0/3388748321 conn(0x7f9f580fe5c0 msgr2=0x7f9f58113920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.759+0000 7f9f5dc5e640 1 -- 172.21.15.160:0/3388748321 shutdown_connections 2024-09-16T08:46:00.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.759+0000 7f9f5dc5e640 1 -- 172.21.15.160:0/3388748321 wait complete. 2024-09-16T08:46:00.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.100+0000 7f78d1353640 1 -- 172.21.15.160:0/1811225655 >> v1:172.21.15.160:6790/0 conn(0x7f78cc10a570 legacy=0x7f78cc10c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:00.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.100+0000 7f78d1353640 1 -- 172.21.15.160:0/1811225655 shutdown_connections 2024-09-16T08:46:00.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.100+0000 7f78d1353640 1 -- 172.21.15.160:0/1811225655 >> 172.21.15.160:0/1811225655 conn(0x7f78cc0fd450 msgr2=0x7f78cc0ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:00.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.100+0000 7f78d1353640 1 -- 172.21.15.160:0/1811225655 shutdown_connections 2024-09-16T08:46:00.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.100+0000 7f78d1353640 1 -- 172.21.15.160:0/1811225655 wait complete. 2024-09-16T08:46:00.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.101+0000 7f78d1353640 1 Processor -- start 2024-09-16T08:46:00.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.101+0000 7f78d1353640 1 -- start start 2024-09-16T08:46:00.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.101+0000 7f78d1353640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f78cc1a7810 con 0x7f78cc10e110 2024-09-16T08:46:00.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.101+0000 7f78d1353640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f78cc1a8a10 con 0x7f78cc10a570 2024-09-16T08:46:00.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.101+0000 7f78d1353640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f78cc1a9c10 con 0x7f78cc101a00 2024-09-16T08:46:00.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.101+0000 7f78cb7fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f78cc10e110 0x7f78cc1a5f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:43326/0 (socket says 172.21.15.160:43326) 2024-09-16T08:46:00.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.101+0000 7f78cb7fe640 1 -- 172.21.15.160:0/601521471 learned_addr learned my addr 172.21.15.160:0/601521471 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:00.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.101+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 671467635 0 0) 0x7f78cc1a7810 con 0x7f78cc10e110 2024-09-16T08:46:00.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.101+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78b4003160 con 0x7f78cc10e110 2024-09-16T08:46:00.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1864915469 0 0) 0x7f78cc1a8a10 con 0x7f78cc10a570 2024-09-16T08:46:00.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78cc1a7810 con 0x7f78cc10a570 2024-09-16T08:46:00.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3048257875 0 0) 0x7f78cc1a9c10 con 0x7f78cc101a00 2024-09-16T08:46:00.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78cc1a8a10 con 0x7f78cc101a00 2024-09-16T08:46:00.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3257074241 0 0) 0x7f78b4003160 con 0x7f78cc10e110 2024-09-16T08:46:00.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f78cc1a9c10 con 0x7f78cc10e110 2024-09-16T08:46:00.872 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3953756130 0 0) 0x7f78cc1a7810 con 0x7f78cc10a570 2024-09-16T08:46:00.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f78b4003160 con 0x7f78cc10a570 2024-09-16T08:46:00.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f78c4001790 con 0x7f78cc10e110 2024-09-16T08:46:00.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f78c00019d0 con 0x7f78cc10a570 2024-09-16T08:46:00.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3491533351 0 0) 0x7f78cc1a8a10 con 0x7f78cc101a00 2024-09-16T08:46:00.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f78cc1a7810 con 0x7f78cc101a00 2024-09-16T08:46:00.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f78bc001830 con 0x7f78cc101a00 2024-09-16T08:46:00.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1160864504 0 0) 0x7f78cc1a9c10 con 0x7f78cc10e110 2024-09-16T08:46:00.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 >> v1:172.21.15.160:6790/0 conn(0x7f78cc101a00 legacy=0x7f78cc10d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:00.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 >> v1:172.21.15.169:6789/0 conn(0x7f78cc10a570 legacy=0x7f78cc1a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:00.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc1aaeb0 con 0x7f78cc10e110 2024-09-16T08:46:00.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78d1353640 1 -- 172.21.15.160:0/601521471 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f78cc1a8c40 con 0x7f78cc10e110 2024-09-16T08:46:00.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78d1353640 1 -- 172.21.15.160:0/601521471 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f78cc1a91a0 con 0x7f78cc10e110 2024-09-16T08:46:00.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f78c40019d0 con 0x7f78cc10e110 2024-09-16T08:46:00.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.102+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f78c4005f70 con 0x7f78cc10e110 2024-09-16T08:46:00.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.103+0000 7f78d1353640 1 -- 172.21.15.160:0/601521471 --> v1:172.21.15.160:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f78b8000fb0 con 0x7f78cc10e110 2024-09-16T08:46:00.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.104+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 17) ==== 103110+0+0 (unknown 251840139 0 0) 0x7f78c401f480 con 0x7f78cc10e110 2024-09-16T08:46:00.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.105+0000 7f78caffd640 1 -- 172.21.15.160:0/601521471 >> v1:172.21.15.160:6800/1286008384 conn(0x7f78b407bc40 legacy=0x7f78b407e100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/1286008384 2024-09-16T08:46:00.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.105+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(63..63 src has 1..63) ==== 5942+0+0 (unknown 3139999450 0 0) 0x7f78c40995e0 con 0x7f78cc10e110 2024-09-16T08:46:00.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.105+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 --> v1:172.21.15.169:6800/1893879009 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78b4089db0 con 0x7f78b40860e0 2024-09-16T08:46:00.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.105+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (unknown 2499814743 0 0) 0x7f78c4099940 con 0x7f78cc10e110 2024-09-16T08:46:00.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.106+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== osd.4 v1:172.21.15.169:6800/1893879009 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f78b4089db0 con 0x7f78b40860e0 2024-09-16T08:46:00.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.120+0000 7f78d1353640 1 -- 172.21.15.160:0/601521471 --> v1:172.21.15.169:6800/1893879009 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f78b8002ca0 con 0x7f78b40860e0 2024-09-16T08:46:00.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.120+0000 7f78b3fff640 1 -- 172.21.15.160:0/601521471 <== osd.4 v1:172.21.15.169:6800/1893879009 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 128125449) 0x7f78b8002ca0 con 0x7f78b40860e0 2024-09-16T08:46:00.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.121+0000 7f78d1353640 1 -- 172.21.15.160:0/601521471 >> v1:172.21.15.169:6800/1893879009 conn(0x7f78b40860e0 legacy=0x7f78b4088540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:00.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.121+0000 7f78d1353640 1 -- 172.21.15.160:0/601521471 >> v1:172.21.15.160:6800/1286008384 conn(0x7f78b407bc40 legacy=0x7f78b407e100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T08:46:00.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.121+0000 7f78d1353640 1 -- 172.21.15.160:0/601521471 >> v1:172.21.15.160:6789/0 conn(0x7f78cc10e110 legacy=0x7f78cc1a5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:00.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.121+0000 7f78cb7fe640 1 -- 172.21.15.160:0/601521471 reap_dead start 2024-09-16T08:46:00.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.121+0000 7f78d1353640 1 -- 172.21.15.160:0/601521471 shutdown_connections 2024-09-16T08:46:00.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.121+0000 7f78d1353640 1 -- 172.21.15.160:0/601521471 >> 172.21.15.160:0/601521471 conn(0x7f78cc0fd450 msgr2=0x7f78cc105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:00.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.121+0000 7f78d1353640 1 -- 172.21.15.160:0/601521471 shutdown_connections 2024-09-16T08:46:00.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:00.121+0000 7f78d1353640 1 -- 172.21.15.160:0/601521471 wait complete. 2024-09-16T08:46:01.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.388+0000 7f831a3e2640 1 -- 172.21.15.160:0/2798455677 >> v1:172.21.15.160:6789/0 conn(0x7f831410e110 legacy=0x7f83141105d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:01.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.388+0000 7f831a3e2640 1 -- 172.21.15.160:0/2798455677 shutdown_connections 2024-09-16T08:46:01.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.388+0000 7f831a3e2640 1 -- 172.21.15.160:0/2798455677 >> 172.21.15.160:0/2798455677 conn(0x7f83140fd450 msgr2=0x7f83140ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:01.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.388+0000 7f831a3e2640 1 -- 172.21.15.160:0/2798455677 shutdown_connections 2024-09-16T08:46:01.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.388+0000 7f831a3e2640 1 -- 172.21.15.160:0/2798455677 wait complete. 2024-09-16T08:46:01.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.389+0000 7f831a3e2640 1 Processor -- start 2024-09-16T08:46:01.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.389+0000 7f831a3e2640 1 -- start start 2024-09-16T08:46:01.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.389+0000 7f831a3e2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f83141a7620 con 0x7f831410a570 2024-09-16T08:46:01.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.389+0000 7f831a3e2640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f83141a8820 con 0x7f831410e110 2024-09-16T08:46:01.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.389+0000 7f831a3e2640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f83141a9a20 con 0x7f8314101a00 2024-09-16T08:46:01.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.389+0000 7f8313fff640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f8314101a00 0x7f831410d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:50930/0 (socket says 172.21.15.160:50930) 2024-09-16T08:46:01.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.389+0000 7f8313fff640 1 -- 172.21.15.160:0/518132106 learned_addr learned my addr 172.21.15.160:0/518132106 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:01.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1835706030 0 0) 0x7f83141a9a20 con 0x7f8314101a00 2024-09-16T08:46:01.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82fc003520 con 0x7f8314101a00 2024-09-16T08:46:01.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 150646620 0 0) 0x7f83141a7620 con 0x7f831410a570 2024-09-16T08:46:01.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83141a9a20 con 0x7f831410a570 2024-09-16T08:46:01.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2965894202 0 0) 0x7f83141a8820 con 0x7f831410e110 2024-09-16T08:46:01.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83141a7620 con 0x7f831410e110 2024-09-16T08:46:01.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2406960334 0 0) 0x7f82fc003520 con 0x7f8314101a00 2024-09-16T08:46:01.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f83141a8820 con 0x7f8314101a00 2024-09-16T08:46:01.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f8304001470 con 0x7f8314101a00 2024-09-16T08:46:01.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 496885225 0 0) 0x7f83141a9a20 con 0x7f831410a570 2024-09-16T08:46:01.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f82fc003520 con 0x7f831410a570 2024-09-16T08:46:01.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f8308001830 con 0x7f831410a570 2024-09-16T08:46:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 37071203 0 0) 0x7f83141a8820 con 0x7f8314101a00 2024-09-16T08:46:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 >> v1:172.21.15.169:6789/0 conn(0x7f831410e110 legacy=0x7f83141a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 >> v1:172.21.15.160:6789/0 conn(0x7f831410a570 legacy=0x7f83141a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83141aacc0 con 0x7f8314101a00 2024-09-16T08:46:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f831a3e2640 1 -- 172.21.15.160:0/518132106 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f83141a9c50 con 0x7f8314101a00 2024-09-16T08:46:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f831a3e2640 1 -- 172.21.15.160:0/518132106 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f83141aa200 con 0x7f8314101a00 2024-09-16T08:46:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f83040016b0 con 0x7f8314101a00 2024-09-16T08:46:01.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.390+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f8304005c60 con 0x7f8314101a00 2024-09-16T08:46:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.391+0000 7f831a3e2640 1 -- 172.21.15.160:0/518132106 --> v1:172.21.15.160:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f8300000fb0 con 0x7f8314101a00 2024-09-16T08:46:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.392+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 17) ==== 103110+0+0 (unknown 251840139 0 0) 0x7f83040038a0 con 0x7f8314101a00 2024-09-16T08:46:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.393+0000 7f83137fe640 1 -- 172.21.15.160:0/518132106 >> v1:172.21.15.160:6800/1286008384 conn(0x7f82fc07b8f0 legacy=0x7f82fc07ddb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/1286008384 2024-09-16T08:46:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.393+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(63..63 src has 1..63) ==== 5942+0+0 (unknown 3139999450 0 0) 0x7f83040991a0 con 0x7f8314101a00 2024-09-16T08:46:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.393+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 --> v1:172.21.15.160:6813/1752204566 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f82fc089a60 con 0x7f82fc085d90 2024-09-16T08:46:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.393+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (unknown 2499814743 0 0) 0x7f8304099580 con 0x7f8314101a00 2024-09-16T08:46:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.394+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 <== osd.3 v1:172.21.15.160:6813/1752204566 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f82fc089a60 con 0x7f82fc085d90 2024-09-16T08:46:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.407+0000 7f831a3e2640 1 -- 172.21.15.160:0/518132106 --> v1:172.21.15.160:6813/1752204566 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8300002ca0 con 0x7f82fc085d90 2024-09-16T08:46:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.408+0000 7f83117fa640 1 -- 172.21.15.160:0/518132106 <== osd.3 v1:172.21.15.160:6813/1752204566 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 3518001440) 0x7f8300002ca0 con 0x7f82fc085d90 2024-09-16T08:46:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.408+0000 7f831a3e2640 1 -- 172.21.15.160:0/518132106 >> v1:172.21.15.160:6813/1752204566 conn(0x7f82fc085d90 legacy=0x7f82fc0881f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.408+0000 7f831a3e2640 1 -- 172.21.15.160:0/518132106 >> v1:172.21.15.160:6800/1286008384 conn(0x7f82fc07b8f0 legacy=0x7f82fc07ddb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T08:46:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.408+0000 7f831a3e2640 1 -- 172.21.15.160:0/518132106 >> v1:172.21.15.160:6790/0 conn(0x7f8314101a00 legacy=0x7f831410d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.408+0000 7f831897f640 1 -- 172.21.15.160:0/518132106 reap_dead start 2024-09-16T08:46:01.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.408+0000 7f831a3e2640 1 -- 172.21.15.160:0/518132106 shutdown_connections 2024-09-16T08:46:01.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.408+0000 7f831a3e2640 1 -- 172.21.15.160:0/518132106 >> 172.21.15.160:0/518132106 conn(0x7f83140fd450 msgr2=0x7f8314110da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:01.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.408+0000 7f831a3e2640 1 -- 172.21.15.160:0/518132106 shutdown_connections 2024-09-16T08:46:01.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:01.408+0000 7f831a3e2640 1 -- 172.21.15.160:0/518132106 wait complete. 2024-09-16T08:46:02.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:02 smithi169 ceph-mon[33465]: Standby manager daemon x restarted 2024-09-16T08:46:02.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:02 smithi169 ceph-mon[33465]: Standby manager daemon x started 2024-09-16T08:46:02.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:02 smithi169 ceph-mon[33465]: from='mgr.? v1:172.21.15.169:0/4252428149' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-16T08:46:02.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:02 smithi169 ceph-mon[33465]: from='mgr.? v1:172.21.15.169:0/4252428149' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-16T08:46:02.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:02 smithi169 ceph-mon[33465]: from='mgr.? v1:172.21.15.169:0/4252428149' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-16T08:46:02.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:02 smithi169 ceph-mon[33465]: from='mgr.? v1:172.21.15.169:0/4252428149' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-16T08:46:02.851 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 08:46:02 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-x[35126]: [16/Sep/2024:08:46:02] ENGINE Bus STARTING 2024-09-16T08:46:02.851 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 08:46:02 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-x[35126]: CherryPy Checker: 2024-09-16T08:46:02.851 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 08:46:02 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-x[35126]: The Application mounted at '' has an empty config. 2024-09-16T08:46:02.851 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 08:46:02 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-x[35126]: 2024-09-16T08:46:02.851 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 08:46:02 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-x[35126]: [16/Sep/2024:08:46:02] ENGINE Serving on http://:::9283 2024-09-16T08:46:02.851 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 08:46:02 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-x[35126]: [16/Sep/2024:08:46:02] ENGINE Bus STARTED 2024-09-16T08:46:02.859 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:02 smithi160 ceph-mon[30137]: Standby manager daemon x restarted 2024-09-16T08:46:02.860 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:02 smithi160 ceph-mon[30137]: Standby manager daemon x started 2024-09-16T08:46:02.860 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:02 smithi160 ceph-mon[30137]: from='mgr.? v1:172.21.15.169:0/4252428149' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-16T08:46:02.860 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:02 smithi160 ceph-mon[30137]: from='mgr.? v1:172.21.15.169:0/4252428149' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-16T08:46:02.860 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:02 smithi160 ceph-mon[30137]: from='mgr.? v1:172.21.15.169:0/4252428149' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-16T08:46:02.860 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:02 smithi160 ceph-mon[30137]: from='mgr.? v1:172.21.15.169:0/4252428149' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-16T08:46:02.860 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:02 smithi160 ceph-mon[40028]: Standby manager daemon x restarted 2024-09-16T08:46:02.860 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:02 smithi160 ceph-mon[40028]: Standby manager daemon x started 2024-09-16T08:46:02.860 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:02 smithi160 ceph-mon[40028]: from='mgr.? v1:172.21.15.169:0/4252428149' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-16T08:46:02.860 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:02 smithi160 ceph-mon[40028]: from='mgr.? v1:172.21.15.169:0/4252428149' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-16T08:46:02.861 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:02 smithi160 ceph-mon[40028]: from='mgr.? v1:172.21.15.169:0/4252428149' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-16T08:46:02.861 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:02 smithi160 ceph-mon[40028]: from='mgr.? v1:172.21.15.169:0/4252428149' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-16T08:46:02.891 INFO:teuthology.orchestra.run.smithi160.stdout:111669149728 2024-09-16T08:46:02.891 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T08:46:03.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.121+0000 7f408d63c640 1 -- 172.21.15.160:0/3654320295 >> v1:172.21.15.169:6789/0 conn(0x7f40881038c0 legacy=0x7f4088103ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:03.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.121+0000 7f408d63c640 1 -- 172.21.15.160:0/3654320295 shutdown_connections 2024-09-16T08:46:03.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.121+0000 7f408d63c640 1 -- 172.21.15.160:0/3654320295 >> 172.21.15.160:0/3654320295 conn(0x7f40880fd4b0 msgr2=0x7f40880ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:03.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.121+0000 7f408d63c640 1 -- 172.21.15.160:0/3654320295 shutdown_connections 2024-09-16T08:46:03.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.121+0000 7f408d63c640 1 -- 172.21.15.160:0/3654320295 wait complete. 2024-09-16T08:46:03.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.122+0000 7f408d63c640 1 Processor -- start 2024-09-16T08:46:03.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.122+0000 7f408d63c640 1 -- start start 2024-09-16T08:46:03.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.122+0000 7f408d63c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f40881a75e0 con 0x7f40881038c0 2024-09-16T08:46:03.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.122+0000 7f408d63c640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f40881a87e0 con 0x7f408810b8a0 2024-09-16T08:46:03.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.122+0000 7f408d63c640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f40881a99e0 con 0x7f408810f3b0 2024-09-16T08:46:03.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.122+0000 7f4086ffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f40881038c0 0x7f408810ade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:43428/0 (socket says 172.21.15.160:43428) 2024-09-16T08:46:03.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.122+0000 7f4086ffd640 1 -- 172.21.15.160:0/3378515850 learned_addr learned my addr 172.21.15.160:0/3378515850 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:03.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.122+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3740651949 0 0) 0x7f40881a75e0 con 0x7f40881038c0 2024-09-16T08:46:03.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.122+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4070003160 con 0x7f40881038c0 2024-09-16T08:46:03.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.122+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1692865994 0 0) 0x7f40881a99e0 con 0x7f408810f3b0 2024-09-16T08:46:03.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.122+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40881a75e0 con 0x7f408810f3b0 2024-09-16T08:46:03.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4080944944 0 0) 0x7f40881a87e0 con 0x7f408810b8a0 2024-09-16T08:46:03.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40881a99e0 con 0x7f408810b8a0 2024-09-16T08:46:03.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 617878078 0 0) 0x7f4070003160 con 0x7f40881038c0 2024-09-16T08:46:03.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f40881a87e0 con 0x7f40881038c0 2024-09-16T08:46:03.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f4078001350 con 0x7f40881038c0 2024-09-16T08:46:03.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2447390645 0 0) 0x7f40881a99e0 con 0x7f408810b8a0 2024-09-16T08:46:03.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4070003160 con 0x7f408810b8a0 2024-09-16T08:46:03.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f407c001880 con 0x7f408810b8a0 2024-09-16T08:46:03.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 936312858 0 0) 0x7f40881a75e0 con 0x7f408810f3b0 2024-09-16T08:46:03.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4070004210 con 0x7f408810f3b0 2024-09-16T08:46:03.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f4080001e70 con 0x7f408810f3b0 2024-09-16T08:46:03.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 342585332 0 0) 0x7f40881a87e0 con 0x7f40881038c0 2024-09-16T08:46:03.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 >> v1:172.21.15.160:6790/0 conn(0x7f408810f3b0 legacy=0x7f40881a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:03.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 >> v1:172.21.15.169:6789/0 conn(0x7f408810b8a0 legacy=0x7f40881a2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:03.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40881aac80 con 0x7f40881038c0 2024-09-16T08:46:03.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f408d63c640 1 -- 172.21.15.160:0/3378515850 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f40881a9c10 con 0x7f40881038c0 2024-09-16T08:46:03.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f408d63c640 1 -- 172.21.15.160:0/3378515850 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f40881aa240 con 0x7f40881038c0 2024-09-16T08:46:03.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f40780037e0 con 0x7f40881038c0 2024-09-16T08:46:03.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.123+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f4078004c00 con 0x7f40881038c0 2024-09-16T08:46:03.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.124+0000 7f408d63c640 1 -- 172.21.15.160:0/3378515850 --> v1:172.21.15.160:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f4074000fb0 con 0x7f40881038c0 2024-09-16T08:46:03.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.125+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 19) ==== 102761+0+0 (unknown 300391685 0 0) 0x7f4078004da0 con 0x7f40881038c0 2024-09-16T08:46:03.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.125+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(64..64 src has 1..64) ==== 6172+0+0 (unknown 440296652 0 0) 0x7f4078098c30 con 0x7f40881038c0 2024-09-16T08:46:03.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.125+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 --> v1:172.21.15.169:6808/3462109725 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40700864d0 con 0x7f40700828c0 2024-09-16T08:46:03.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.125+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (unknown 2884312405 0 0) 0x7f4078099040 con 0x7f40881038c0 2024-09-16T08:46:03.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.127+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== osd.6 v1:172.21.15.169:6808/3462109725 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f40700864d0 con 0x7f40700828c0 2024-09-16T08:46:03.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.140+0000 7f408d63c640 1 -- 172.21.15.160:0/3378515850 --> v1:172.21.15.169:6808/3462109725 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4074002ca0 con 0x7f40700828c0 2024-09-16T08:46:03.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.140+0000 7f406ffff640 1 -- 172.21.15.160:0/3378515850 <== osd.6 v1:172.21.15.169:6808/3462109725 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 881677591) 0x7f4074002ca0 con 0x7f40700828c0 2024-09-16T08:46:03.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.141+0000 7f408d63c640 1 -- 172.21.15.160:0/3378515850 >> v1:172.21.15.169:6808/3462109725 conn(0x7f40700828c0 legacy=0x7f4070084d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:03.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.141+0000 7f408d63c640 1 -- 172.21.15.160:0/3378515850 >> v1:172.21.15.160:6789/0 conn(0x7f40881038c0 legacy=0x7f408810ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:03.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.141+0000 7f408d63c640 1 -- 172.21.15.160:0/3378515850 shutdown_connections 2024-09-16T08:46:03.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.141+0000 7f408d63c640 1 -- 172.21.15.160:0/3378515850 >> 172.21.15.160:0/3378515850 conn(0x7f40880fd4b0 msgr2=0x7f40880fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:03.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.141+0000 7f408d63c640 1 -- 172.21.15.160:0/3378515850 shutdown_connections 2024-09-16T08:46:03.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.141+0000 7f408d63c640 1 -- 172.21.15.160:0/3378515850 wait complete. 2024-09-16T08:46:03.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.497+0000 7f793c817640 1 -- 172.21.15.160:0/2316940549 >> v1:172.21.15.160:6789/0 conn(0x7f793810a570 legacy=0x7f793810c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:03.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.497+0000 7f793c817640 1 -- 172.21.15.160:0/2316940549 shutdown_connections 2024-09-16T08:46:03.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.497+0000 7f793c817640 1 -- 172.21.15.160:0/2316940549 >> 172.21.15.160:0/2316940549 conn(0x7f79380fd450 msgr2=0x7f79380ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:03.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.497+0000 7f793c817640 1 -- 172.21.15.160:0/2316940549 shutdown_connections 2024-09-16T08:46:03.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.497+0000 7f793c817640 1 -- 172.21.15.160:0/2316940549 wait complete. 2024-09-16T08:46:03.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.499+0000 7f793c817640 1 Processor -- start 2024-09-16T08:46:03.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.499+0000 7f793c817640 1 -- start start 2024-09-16T08:46:03.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.499+0000 7f793c817640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f793810ddf0 con 0x7f7938101a00 2024-09-16T08:46:03.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.499+0000 7f793c817640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f79381a8cd0 con 0x7f793810e110 2024-09-16T08:46:03.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.499+0000 7f793c817640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f79381a9eb0 con 0x7f793810a570 2024-09-16T08:46:03.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.499+0000 7f793659c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f7938101a00 0x7f793806ac30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:43382/0 (socket says 172.21.15.160:43382) 2024-09-16T08:46:03.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.499+0000 7f793659c640 1 -- 172.21.15.160:0/1122428561 learned_addr learned my addr 172.21.15.160:0/1122428561 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:03.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.499+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3681678350 0 0) 0x7f79381a9eb0 con 0x7f793810a570 2024-09-16T08:46:03.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.499+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7920003520 con 0x7f793810a570 2024-09-16T08:46:03.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.499+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3006769111 0 0) 0x7f793810ddf0 con 0x7f7938101a00 2024-09-16T08:46:03.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.499+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79381a9eb0 con 0x7f7938101a00 2024-09-16T08:46:03.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.499+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2761333832 0 0) 0x7f79381a8cd0 con 0x7f793810e110 2024-09-16T08:46:03.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.499+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f793810ddf0 con 0x7f793810e110 2024-09-16T08:46:03.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 45239602 0 0) 0x7f7920003520 con 0x7f793810a570 2024-09-16T08:46:03.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f79381a8cd0 con 0x7f793810a570 2024-09-16T08:46:03.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f792c0014b0 con 0x7f793810a570 2024-09-16T08:46:03.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4105901170 0 0) 0x7f79381a9eb0 con 0x7f7938101a00 2024-09-16T08:46:03.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7920003520 con 0x7f7938101a00 2024-09-16T08:46:03.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f7928001970 con 0x7f7938101a00 2024-09-16T08:46:03.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3419977445 0 0) 0x7f793810ddf0 con 0x7f793810e110 2024-09-16T08:46:03.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f79381a9eb0 con 0x7f793810e110 2024-09-16T08:46:03.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f7930001bf0 con 0x7f793810e110 2024-09-16T08:46:03.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4239311010 0 0) 0x7f79381a8cd0 con 0x7f793810a570 2024-09-16T08:46:03.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 >> v1:172.21.15.169:6789/0 conn(0x7f793810e110 legacy=0x7f79381a65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:03.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 >> v1:172.21.15.160:6789/0 conn(0x7f7938101a00 legacy=0x7f793806ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:03.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79381ab090 con 0x7f793810a570 2024-09-16T08:46:03.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f793c817640 1 -- 172.21.15.160:0/1122428561 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f79381a7d20 con 0x7f793810a570 2024-09-16T08:46:03.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f793c817640 1 -- 172.21.15.160:0/1122428561 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f79381a8230 con 0x7f793810a570 2024-09-16T08:46:03.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f792c003800 con 0x7f793810a570 2024-09-16T08:46:03.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.500+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f792c004b60 con 0x7f793810a570 2024-09-16T08:46:03.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.501+0000 7f793c817640 1 -- 172.21.15.160:0/1122428561 --> v1:172.21.15.160:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f7918000fb0 con 0x7f793810a570 2024-09-16T08:46:03.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.502+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 17) ==== 103110+0+0 (unknown 251840139 0 0) 0x7f792c01e050 con 0x7f793810a570 2024-09-16T08:46:03.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.502+0000 7f793659c640 1 -- 172.21.15.160:0/1122428561 >> v1:172.21.15.160:6800/1286008384 conn(0x7f792007bc80 legacy=0x7f792007e140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.160:6800/1286008384 2024-09-16T08:46:03.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.503+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(63..63 src has 1..63) ==== 5942+0+0 (unknown 3139999450 0 0) 0x7f792c098020 con 0x7f793810a570 2024-09-16T08:46:03.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.503+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 --> v1:172.21.15.160:6809/2138225188 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f792c0040e0 con 0x7f79200861a0 2024-09-16T08:46:03.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.503+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (unknown 2499814743 0 0) 0x7f792c098430 con 0x7f793810a570 2024-09-16T08:46:03.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.504+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== osd.2 v1:172.21.15.160:6809/2138225188 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f792c0040e0 con 0x7f79200861a0 2024-09-16T08:46:03.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.517+0000 7f793c817640 1 -- 172.21.15.160:0/1122428561 --> v1:172.21.15.160:6809/2138225188 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7918002ca0 con 0x7f79200861a0 2024-09-16T08:46:03.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.517+0000 7f79277fe640 1 -- 172.21.15.160:0/1122428561 <== osd.2 v1:172.21.15.160:6809/2138225188 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 1652615586) 0x7f7918002ca0 con 0x7f79200861a0 2024-09-16T08:46:03.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.518+0000 7f793c817640 1 -- 172.21.15.160:0/1122428561 >> v1:172.21.15.160:6809/2138225188 conn(0x7f79200861a0 legacy=0x7f7920088600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:03.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.518+0000 7f793c817640 1 -- 172.21.15.160:0/1122428561 >> v1:172.21.15.160:6800/1286008384 conn(0x7f792007bc80 legacy=0x7f792007e140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T08:46:03.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.518+0000 7f793c817640 1 -- 172.21.15.160:0/1122428561 >> v1:172.21.15.160:6790/0 conn(0x7f793810a570 legacy=0x7f793806b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:03.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.518+0000 7f7936d9d640 1 -- 172.21.15.160:0/1122428561 reap_dead start 2024-09-16T08:46:03.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.518+0000 7f793c817640 1 -- 172.21.15.160:0/1122428561 shutdown_connections 2024-09-16T08:46:03.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.518+0000 7f793c817640 1 -- 172.21.15.160:0/1122428561 >> 172.21.15.160:0/1122428561 conn(0x7f79380fd450 msgr2=0x7f7938100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:03.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.518+0000 7f793c817640 1 -- 172.21.15.160:0/1122428561 shutdown_connections 2024-09-16T08:46:03.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:02.518+0000 7f793c817640 1 -- 172.21.15.160:0/1122428561 wait complete. 2024-09-16T08:46:03.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.644+0000 7f92bc67e640 1 -- 172.21.15.160:0/3069155228 >> v1:172.21.15.169:6789/0 conn(0x7f92b4111460 legacy=0x7f92b4113920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:03.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.645+0000 7f92bc67e640 1 -- 172.21.15.160:0/3069155228 shutdown_connections 2024-09-16T08:46:03.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.645+0000 7f92bc67e640 1 -- 172.21.15.160:0/3069155228 >> 172.21.15.160:0/3069155228 conn(0x7f92b40fd450 msgr2=0x7f92b40ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:03.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.645+0000 7f92bc67e640 1 -- 172.21.15.160:0/3069155228 shutdown_connections 2024-09-16T08:46:03.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.645+0000 7f92bc67e640 1 -- 172.21.15.160:0/3069155228 wait complete. 2024-09-16T08:46:03.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.645+0000 7f92bc67e640 1 Processor -- start 2024-09-16T08:46:03.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.645+0000 7f92bc67e640 1 -- start start 2024-09-16T08:46:03.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.646+0000 7f92bc67e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f92b410e910 con 0x7f92b4111460 2024-09-16T08:46:03.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.646+0000 7f92bac1b640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f92b4111460 0x7f92b41a8340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:43478/0 (socket says 172.21.15.160:43478) 2024-09-16T08:46:03.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.646+0000 7f92bac1b640 1 -- 172.21.15.160:0/1326005679 learned_addr learned my addr 172.21.15.160:0/1326005679 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:03.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.646+0000 7f92ba41a640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f92b4101600 0x7f92b410dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:51016/0 (socket says 172.21.15.160:51016) 2024-09-16T08:46:03.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.646+0000 7f92bc67e640 1 -- 172.21.15.160:0/1326005679 --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f92b41aaa70 con 0x7f92b41020f0 2024-09-16T08:46:03.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.646+0000 7f92bc67e640 1 -- 172.21.15.160:0/1326005679 --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f92b41abc50 con 0x7f92b4101600 2024-09-16T08:46:03.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.646+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3530266734 0 0) 0x7f92b41abc50 con 0x7f92b4101600 2024-09-16T08:46:03.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.646+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92a0003160 con 0x7f92b4101600 2024-09-16T08:46:03.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.646+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3198984531 0 0) 0x7f92b410e910 con 0x7f92b4111460 2024-09-16T08:46:03.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.646+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92b41abc50 con 0x7f92b4111460 2024-09-16T08:46:03.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.646+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 193615712 0 0) 0x7f92a0003160 con 0x7f92b4101600 2024-09-16T08:46:03.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.646+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f92b410e910 con 0x7f92b4101600 2024-09-16T08:46:03.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.646+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f92b0001af0 con 0x7f92b4101600 2024-09-16T08:46:03.692 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.647+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1037053560 0 0) 0x7f92b41abc50 con 0x7f92b4111460 2024-09-16T08:46:03.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.647+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f92a0003160 con 0x7f92b4111460 2024-09-16T08:46:03.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.647+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f92ac001ae0 con 0x7f92b4111460 2024-09-16T08:46:03.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.647+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 344360 0 0) 0x7f92b410e910 con 0x7f92b4101600 2024-09-16T08:46:03.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.647+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 >> v1:172.21.15.169:6789/0 conn(0x7f92b41020f0 legacy=0x7f92b41a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:03.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.647+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 >> v1:172.21.15.160:6789/0 conn(0x7f92b4111460 legacy=0x7f92b41a8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:03.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.647+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b41ace80 con 0x7f92b4101600 2024-09-16T08:46:03.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.647+0000 7f92bc67e640 1 -- 172.21.15.160:0/1326005679 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f92b41a9ac0 con 0x7f92b4101600 2024-09-16T08:46:03.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.647+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f92b0001d10 con 0x7f92b4101600 2024-09-16T08:46:03.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.647+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f92b0005c70 con 0x7f92b4101600 2024-09-16T08:46:03.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.647+0000 7f92bc67e640 1 -- 172.21.15.160:0/1326005679 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f92b41aa070 con 0x7f92b4101600 2024-09-16T08:46:03.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.648+0000 7f92bc67e640 1 -- 172.21.15.160:0/1326005679 --> v1:172.21.15.160:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f92b406b240 con 0x7f92b4101600 2024-09-16T08:46:03.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.649+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 19) ==== 102761+0+0 (unknown 300391685 0 0) 0x7f92b00038a0 con 0x7f92b4101600 2024-09-16T08:46:03.693 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.650+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(64..64 src has 1..64) ==== 6172+0+0 (unknown 440296652 0 0) 0x7f92b0098ed0 con 0x7f92b4101600 2024-09-16T08:46:03.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.650+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 --> v1:172.21.15.169:6804/4217518758 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92a0085d80 con 0x7f92a0082170 2024-09-16T08:46:03.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.650+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (unknown 2884312405 0 0) 0x7f92b0099330 con 0x7f92b4101600 2024-09-16T08:46:03.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.651+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 <== osd.5 v1:172.21.15.169:6804/4217518758 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f92a0085d80 con 0x7f92a0082170 2024-09-16T08:46:03.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.678+0000 7f92bc67e640 1 -- 172.21.15.160:0/1326005679 --> v1:172.21.15.169:6804/4217518758 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f92b41aa650 con 0x7f92a0082170 2024-09-16T08:46:03.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.678+0000 7f92ab7fe640 1 -- 172.21.15.160:0/1326005679 <== osd.5 v1:172.21.15.169:6804/4217518758 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 3847739399) 0x7f92b41aa650 con 0x7f92a0082170 2024-09-16T08:46:03.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.678+0000 7f92bc67e640 1 -- 172.21.15.160:0/1326005679 >> v1:172.21.15.169:6804/4217518758 conn(0x7f92a0082170 legacy=0x7f92a00845b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:03.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.679+0000 7f92bc67e640 1 -- 172.21.15.160:0/1326005679 >> v1:172.21.15.160:6790/0 conn(0x7f92b4101600 legacy=0x7f92b410dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:03.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.679+0000 7f92bc67e640 1 -- 172.21.15.160:0/1326005679 shutdown_connections 2024-09-16T08:46:03.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.679+0000 7f92bc67e640 1 -- 172.21.15.160:0/1326005679 >> 172.21.15.160:0/1326005679 conn(0x7f92b40fd450 msgr2=0x7f92b40ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:03.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.679+0000 7f92bc67e640 1 -- 172.21.15.160:0/1326005679 shutdown_connections 2024-09-16T08:46:03.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:03.679+0000 7f92bc67e640 1 -- 172.21.15.160:0/1326005679 wait complete. 2024-09-16T08:46:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-16T08:46:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: Active manager daemon y restarted 2024-09-16T08:46:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: Activating manager daemon y 2024-09-16T08:46:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T08:46:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: mgrmap e19: y(active, starting, since 0.00888764s), standbys: x 2024-09-16T08:46:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:46:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:46:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:46:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-16T08:46:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-16T08:46:03.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:46:03.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:46:03.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:46:03.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:46:03.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:46:03.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:46:03.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:46:03.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T08:46:03.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-16T08:46:03.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-16T08:46:03.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-16T08:46:03.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: Manager daemon y is now available 2024-09-16T08:46:03.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:46:03.852 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:03.852 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:46:03.852 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:46:03.852 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-16T08:46:03.852 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-16T08:46:03.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-16T08:46:03.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: Active manager daemon y restarted 2024-09-16T08:46:03.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: Activating manager daemon y 2024-09-16T08:46:03.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T08:46:03.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: mgrmap e19: y(active, starting, since 0.00888764s), standbys: x 2024-09-16T08:46:03.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:46:03.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:46:03.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:46:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-16T08:46:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-16T08:46:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:46:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:46:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:46:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:46:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:46:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:46:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:46:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T08:46:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-16T08:46:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-16T08:46:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-16T08:46:03.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: Manager daemon y is now available 2024-09-16T08:46:03.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:46:03.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:03.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:46:03.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:46:03.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-16T08:46:03.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-16T08:46:03.950 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-16T08:46:03.950 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: Active manager daemon y restarted 2024-09-16T08:46:03.950 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: Activating manager daemon y 2024-09-16T08:46:03.950 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T08:46:03.950 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: mgrmap e19: y(active, starting, since 0.00888764s), standbys: x 2024-09-16T08:46:03.951 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T08:46:03.951 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T08:46:03.951 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T08:46:03.951 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-16T08:46:03.951 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-16T08:46:03.951 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:46:03.951 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:46:03.951 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:46:03.951 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:46:03.951 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:46:03.951 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:46:03.951 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T08:46:03.951 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T08:46:03.952 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-16T08:46:03.952 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-16T08:46:03.952 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-16T08:46:03.952 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: Manager daemon y is now available 2024-09-16T08:46:03.952 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:46:03.952 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:03.952 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:46:03.952 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:46:03.952 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-16T08:46:03.952 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-16T08:46:04.298 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:46:04] ENGINE Bus STARTING 2024-09-16T08:46:04.298 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: CherryPy Checker: 2024-09-16T08:46:04.298 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: The Application mounted at '' has an empty config. 2024-09-16T08:46:04.298 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: 2024-09-16T08:46:04.298 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:46:04] ENGINE Serving on http://:::9283 2024-09-16T08:46:04.298 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:46:04] ENGINE Bus STARTED 2024-09-16T08:46:04.298 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: 2024-09-16T08:46:04.297+0000 7f834b83b640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:08:46:04] ENGINE Error in HTTPServer.serve 2024-09-16T08:46:04.299 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: Traceback (most recent call last): 2024-09-16T08:46:04.299 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:46:04.299 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: self._connections.run(self.expiration_interval) 2024-09-16T08:46:04.299 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:46:04.299 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: self._run(expiration_interval) 2024-09-16T08:46:04.299 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:46:04.299 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:46:04.299 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:46:04.299 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:46:04.299 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:46:04.681 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: s = self.context.wrap_socket( 2024-09-16T08:46:04.681 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:46:04.681 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: return self.sslsocket_class._create( 2024-09-16T08:46:04.681 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:46:04.681 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: self.do_handshake() 2024-09-16T08:46:04.681 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:46:04.682 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: self._sslobj.do_handshake() 2024-09-16T08:46:04.682 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:46:04.682 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: 2024-09-16T08:46:05.005 INFO:teuthology.orchestra.run.smithi160.stdout:77309411367 2024-09-16T08:46:05.006 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T08:46:05.030 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:46:04 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:46:05.136 INFO:teuthology.orchestra.run.smithi160.stdout:55834574891 2024-09-16T08:46:05.136 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T08:46:05.283 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:05 smithi169 ceph-mon[33465]: mgrmap e20: y(active, since 1.01477s), standbys: x 2024-09-16T08:46:05.283 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:05 smithi169 ceph-mon[33465]: [16/Sep/2024:08:46:04] ENGINE Bus STARTING 2024-09-16T08:46:05.283 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:05 smithi169 ceph-mon[33465]: [16/Sep/2024:08:46:04] ENGINE Serving on https://172.21.15.160:7150 2024-09-16T08:46:05.284 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:05 smithi169 ceph-mon[33465]: [16/Sep/2024:08:46:04] ENGINE Error in HTTPServer.serve 2024-09-16T08:46:05.284 INFO:journalctl@ceph.mon.b.smithi169.stdout: Traceback (most recent call last): 2024-09-16T08:46:05.284 INFO:journalctl@ceph.mon.b.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:46:05.284 INFO:journalctl@ceph.mon.b.smithi169.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:46:05.284 INFO:journalctl@ceph.mon.b.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:46:05.284 INFO:journalctl@ceph.mon.b.smithi169.stdout: self._run(expiration_interval) 2024-09-16T08:46:05.284 INFO:journalctl@ceph.mon.b.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:46:05.284 INFO:journalctl@ceph.mon.b.smithi169.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:46:05.284 INFO:journalctl@ceph.mon.b.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:46:05.284 INFO:journalctl@ceph.mon.b.smithi169.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:46:05.284 INFO:journalctl@ceph.mon.b.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:46:05.285 INFO:journalctl@ceph.mon.b.smithi169.stdout: s = self.context.wrap_socket( 2024-09-16T08:46:05.285 INFO:journalctl@ceph.mon.b.smithi169.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:46:05.285 INFO:journalctl@ceph.mon.b.smithi169.stdout: return self.sslsocket_class._create( 2024-09-16T08:46:05.285 INFO:journalctl@ceph.mon.b.smithi169.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:46:05.285 INFO:journalctl@ceph.mon.b.smithi169.stdout: self.do_handshake() 2024-09-16T08:46:05.285 INFO:journalctl@ceph.mon.b.smithi169.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:46:05.285 INFO:journalctl@ceph.mon.b.smithi169.stdout: self._sslobj.do_handshake() 2024-09-16T08:46:05.285 INFO:journalctl@ceph.mon.b.smithi169.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:46:05.285 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:05 smithi169 ceph-mon[33465]: [16/Sep/2024:08:46:04] ENGINE Serving on http://172.21.15.160:8765 2024-09-16T08:46:05.285 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:05 smithi169 ceph-mon[33465]: [16/Sep/2024:08:46:04] ENGINE Bus STARTED 2024-09-16T08:46:05.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:46:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:46:05.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:05 smithi160 ceph-mon[30137]: mgrmap e20: y(active, since 1.01477s), standbys: x 2024-09-16T08:46:05.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:05 smithi160 ceph-mon[30137]: [16/Sep/2024:08:46:04] ENGINE Bus STARTING 2024-09-16T08:46:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:05 smithi160 ceph-mon[30137]: [16/Sep/2024:08:46:04] ENGINE Serving on https://172.21.15.160:7150 2024-09-16T08:46:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:05 smithi160 ceph-mon[30137]: [16/Sep/2024:08:46:04] ENGINE Error in HTTPServer.serve 2024-09-16T08:46:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: Traceback (most recent call last): 2024-09-16T08:46:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:46:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:46:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:46:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: self._run(expiration_interval) 2024-09-16T08:46:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:46:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:46:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:46:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:46:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:46:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout: s = self.context.wrap_socket( 2024-09-16T08:46:05.449 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:46:05.449 INFO:journalctl@ceph.mon.a.smithi160.stdout: return self.sslsocket_class._create( 2024-09-16T08:46:05.449 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:46:05.449 INFO:journalctl@ceph.mon.a.smithi160.stdout: self.do_handshake() 2024-09-16T08:46:05.449 INFO:journalctl@ceph.mon.a.smithi160.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:46:05.449 INFO:journalctl@ceph.mon.a.smithi160.stdout: self._sslobj.do_handshake() 2024-09-16T08:46:05.449 INFO:journalctl@ceph.mon.a.smithi160.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:46:05.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:05 smithi160 ceph-mon[30137]: [16/Sep/2024:08:46:04] ENGINE Serving on http://172.21.15.160:8765 2024-09-16T08:46:05.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:05 smithi160 ceph-mon[30137]: [16/Sep/2024:08:46:04] ENGINE Bus STARTED 2024-09-16T08:46:05.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:05 smithi160 ceph-mon[40028]: mgrmap e20: y(active, since 1.01477s), standbys: x 2024-09-16T08:46:05.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:05 smithi160 ceph-mon[40028]: [16/Sep/2024:08:46:04] ENGINE Bus STARTING 2024-09-16T08:46:05.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:05 smithi160 ceph-mon[40028]: [16/Sep/2024:08:46:04] ENGINE Serving on https://172.21.15.160:7150 2024-09-16T08:46:05.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:05 smithi160 ceph-mon[40028]: [16/Sep/2024:08:46:04] ENGINE Error in HTTPServer.serve 2024-09-16T08:46:05.450 INFO:journalctl@ceph.mon.c.smithi160.stdout: Traceback (most recent call last): 2024-09-16T08:46:05.450 INFO:journalctl@ceph.mon.c.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:46:05.450 INFO:journalctl@ceph.mon.c.smithi160.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:46:05.450 INFO:journalctl@ceph.mon.c.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:46:05.450 INFO:journalctl@ceph.mon.c.smithi160.stdout: self._run(expiration_interval) 2024-09-16T08:46:05.450 INFO:journalctl@ceph.mon.c.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:46:05.450 INFO:journalctl@ceph.mon.c.smithi160.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:46:05.450 INFO:journalctl@ceph.mon.c.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:46:05.450 INFO:journalctl@ceph.mon.c.smithi160.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:46:05.450 INFO:journalctl@ceph.mon.c.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:46:05.450 INFO:journalctl@ceph.mon.c.smithi160.stdout: s = self.context.wrap_socket( 2024-09-16T08:46:05.450 INFO:journalctl@ceph.mon.c.smithi160.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:46:05.450 INFO:journalctl@ceph.mon.c.smithi160.stdout: return self.sslsocket_class._create( 2024-09-16T08:46:05.451 INFO:journalctl@ceph.mon.c.smithi160.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:46:05.451 INFO:journalctl@ceph.mon.c.smithi160.stdout: self.do_handshake() 2024-09-16T08:46:05.451 INFO:journalctl@ceph.mon.c.smithi160.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:46:05.451 INFO:journalctl@ceph.mon.c.smithi160.stdout: self._sslobj.do_handshake() 2024-09-16T08:46:05.451 INFO:journalctl@ceph.mon.c.smithi160.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:46:05.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:05 smithi160 ceph-mon[40028]: [16/Sep/2024:08:46:04] ENGINE Serving on http://172.21.15.160:8765 2024-09-16T08:46:05.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:05 smithi160 ceph-mon[40028]: [16/Sep/2024:08:46:04] ENGINE Bus STARTED 2024-09-16T08:46:05.575 INFO:teuthology.orchestra.run.smithi160.stdout:137438953499 2024-09-16T08:46:05.576 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T08:46:06.041 INFO:teuthology.orchestra.run.smithi160.stdout:188978561037 2024-09-16T08:46:06.041 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-16T08:46:06.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:06.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:06 smithi169 ceph-mon[33465]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-16T08:46:06.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:06 smithi169 ceph-mon[33465]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:46:06.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.231+0000 7f909b9c9640 1 -- 172.21.15.160:0/1462365680 >> v1:172.21.15.160:6790/0 conn(0x7f9094102b40 legacy=0x7f9094102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:06.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.231+0000 7f909b9c9640 1 -- 172.21.15.160:0/1462365680 shutdown_connections 2024-09-16T08:46:06.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.231+0000 7f909b9c9640 1 -- 172.21.15.160:0/1462365680 >> 172.21.15.160:0/1462365680 conn(0x7f90940fe590 msgr2=0x7f90941009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:06.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.231+0000 7f909b9c9640 1 -- 172.21.15.160:0/1462365680 shutdown_connections 2024-09-16T08:46:06.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.231+0000 7f909b9c9640 1 -- 172.21.15.160:0/1462365680 wait complete. 2024-09-16T08:46:06.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.232+0000 7f909b9c9640 1 Processor -- start 2024-09-16T08:46:06.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.232+0000 7f909b9c9640 1 -- start start 2024-09-16T08:46:06.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.232+0000 7f909b9c9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f90941a76f0 con 0x7f909410b6b0 2024-09-16T08:46:06.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.232+0000 7f909b9c9640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f90941a88f0 con 0x7f909410f250 2024-09-16T08:46:06.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.232+0000 7f909b9c9640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f90941a9af0 con 0x7f9094102b40 2024-09-16T08:46:06.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.232+0000 7f9099765640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f9094102b40 0x7f909410e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:51042/0 (socket says 172.21.15.160:51042) 2024-09-16T08:46:06.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.232+0000 7f9099765640 1 -- 172.21.15.160:0/1764515534 learned_addr learned my addr 172.21.15.160:0/1764515534 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:06.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.232+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 115393621 0 0) 0x7f90941a9af0 con 0x7f9094102b40 2024-09-16T08:46:06.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.232+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9080003160 con 0x7f9094102b40 2024-09-16T08:46:06.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.232+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3053606264 0 0) 0x7f90941a76f0 con 0x7f909410b6b0 2024-09-16T08:46:06.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.232+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90941a9af0 con 0x7f909410b6b0 2024-09-16T08:46:06.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.232+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2210013727 0 0) 0x7f90941a88f0 con 0x7f909410f250 2024-09-16T08:46:06.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.232+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90941a76f0 con 0x7f909410f250 2024-09-16T08:46:06.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1152199702 0 0) 0x7f9080003160 con 0x7f9094102b40 2024-09-16T08:46:06.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f90941a88f0 con 0x7f9094102b40 2024-09-16T08:46:06.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f90900018b0 con 0x7f9094102b40 2024-09-16T08:46:06.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4279131897 0 0) 0x7f90941a9af0 con 0x7f909410b6b0 2024-09-16T08:46:06.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9080003160 con 0x7f909410b6b0 2024-09-16T08:46:06.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f90840010e0 con 0x7f909410b6b0 2024-09-16T08:46:06.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2320885139 0 0) 0x7f90941a76f0 con 0x7f909410f250 2024-09-16T08:46:06.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f90941a9af0 con 0x7f909410f250 2024-09-16T08:46:06.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f908c001bf0 con 0x7f909410f250 2024-09-16T08:46:06.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1208934936 0 0) 0x7f90941a88f0 con 0x7f9094102b40 2024-09-16T08:46:06.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 >> v1:172.21.15.169:6789/0 conn(0x7f909410f250 legacy=0x7f90941a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:06.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 >> v1:172.21.15.160:6789/0 conn(0x7f909410b6b0 legacy=0x7f90941a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:06.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90941aad90 con 0x7f9094102b40 2024-09-16T08:46:06.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f909b9c9640 1 -- 172.21.15.160:0/1764515534 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f90941a9d20 con 0x7f9094102b40 2024-09-16T08:46:06.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f909b9c9640 1 -- 172.21.15.160:0/1764515534 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f90941aa350 con 0x7f9094102b40 2024-09-16T08:46:06.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f9090001ad0 con 0x7f9094102b40 2024-09-16T08:46:06.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.233+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f9090004c60 con 0x7f9094102b40 2024-09-16T08:46:06.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.234+0000 7f909b9c9640 1 -- 172.21.15.160:0/1764515534 --> v1:172.21.15.160:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f9078000fb0 con 0x7f9094102b40 2024-09-16T08:46:06.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.234+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 21) ==== 103047+0+0 (unknown 1719902048 0 0) 0x7f9090004e00 con 0x7f9094102b40 2024-09-16T08:46:06.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.235+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(64..64 src has 1..64) ==== 6172+0+0 (unknown 440296652 0 0) 0x7f9090099280 con 0x7f9094102b40 2024-09-16T08:46:06.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.235+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 --> v1:172.21.15.169:6812/1196718305 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9084001d30 con 0x7f9080086430 2024-09-16T08:46:06.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.235+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (unknown 2884312405 0 0) 0x7f9090099680 con 0x7f9094102b40 2024-09-16T08:46:06.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.236+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== osd.7 v1:172.21.15.169:6812/1196718305 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f9084001d30 con 0x7f9080086430 2024-09-16T08:46:06.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.253+0000 7f909b9c9640 1 -- 172.21.15.160:0/1764515534 --> v1:172.21.15.169:6812/1196718305 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9078002ca0 con 0x7f9080086430 2024-09-16T08:46:06.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.253+0000 7f908a7fc640 1 -- 172.21.15.160:0/1764515534 <== osd.7 v1:172.21.15.169:6812/1196718305 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 106851819) 0x7f9078002ca0 con 0x7f9080086430 2024-09-16T08:46:06.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.254+0000 7f909b9c9640 1 -- 172.21.15.160:0/1764515534 >> v1:172.21.15.169:6812/1196718305 conn(0x7f9080086430 legacy=0x7f9080088870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:06.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.254+0000 7f909b9c9640 1 -- 172.21.15.160:0/1764515534 >> v1:172.21.15.160:6800/2859204856 conn(0x7f908007bc40 legacy=0x7f908007e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:06.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.254+0000 7f909b9c9640 1 -- 172.21.15.160:0/1764515534 >> v1:172.21.15.160:6790/0 conn(0x7f9094102b40 legacy=0x7f909410e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:06.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.254+0000 7f9099f66640 1 -- 172.21.15.160:0/1764515534 reap_dead start 2024-09-16T08:46:06.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.254+0000 7f909b9c9640 1 -- 172.21.15.160:0/1764515534 shutdown_connections 2024-09-16T08:46:06.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.254+0000 7f909b9c9640 1 -- 172.21.15.160:0/1764515534 >> 172.21.15.160:0/1764515534 conn(0x7f90940fe590 msgr2=0x7f9094111710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:06.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.254+0000 7f909b9c9640 1 -- 172.21.15.160:0/1764515534 shutdown_connections 2024-09-16T08:46:06.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:06.254+0000 7f909b9c9640 1 -- 172.21.15.160:0/1764515534 wait complete. 2024-09-16T08:46:06.263 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:06.263 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:06 smithi160 ceph-mon[30137]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-16T08:46:06.264 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:06 smithi160 ceph-mon[30137]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:46:06.264 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:06.264 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:06 smithi160 ceph-mon[40028]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-16T08:46:06.264 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:06 smithi160 ceph-mon[40028]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:46:06.962 INFO:teuthology.orchestra.run.smithi160.stdout:214748364809 2024-09-16T08:46:06.963 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-16T08:46:07.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.110+0000 7f96a9287640 1 -- 172.21.15.160:0/4033130572 >> v1:172.21.15.160:6789/0 conn(0x7f96a410e110 legacy=0x7f96a41105d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:07.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.110+0000 7f96a9287640 1 -- 172.21.15.160:0/4033130572 shutdown_connections 2024-09-16T08:46:07.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.110+0000 7f96a9287640 1 -- 172.21.15.160:0/4033130572 >> 172.21.15.160:0/4033130572 conn(0x7f96a40fd450 msgr2=0x7f96a40ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:07.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.110+0000 7f96a9287640 1 -- 172.21.15.160:0/4033130572 shutdown_connections 2024-09-16T08:46:07.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.110+0000 7f96a9287640 1 -- 172.21.15.160:0/4033130572 wait complete. 2024-09-16T08:46:07.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.111+0000 7f96a9287640 1 Processor -- start 2024-09-16T08:46:07.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.111+0000 7f96a9287640 1 -- start start 2024-09-16T08:46:07.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.111+0000 7f96a9287640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f96a41a7760 con 0x7f96a4101a00 2024-09-16T08:46:07.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.111+0000 7f96a9287640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f96a41a8960 con 0x7f96a410e110 2024-09-16T08:46:07.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.111+0000 7f96a9287640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f96a41a9b60 con 0x7f96a410a570 2024-09-16T08:46:07.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f96a2ffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f96a4101a00 0x7f96a410d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:59190/0 (socket says 172.21.15.160:59190) 2024-09-16T08:46:07.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f96a2ffd640 1 -- 172.21.15.160:0/2193571659 learned_addr learned my addr 172.21.15.160:0/2193571659 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:07.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f96a27fc640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f96a410a570 0x7f96a41a2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:55594/0 (socket says 172.21.15.160:55594) 2024-09-16T08:46:07.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2411397221 0 0) 0x7f96a41a7760 con 0x7f96a4101a00 2024-09-16T08:46:07.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f968c003160 con 0x7f96a4101a00 2024-09-16T08:46:07.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3159587995 0 0) 0x7f96a41a9b60 con 0x7f96a410a570 2024-09-16T08:46:07.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96a41a7760 con 0x7f96a410a570 2024-09-16T08:46:07.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1587908587 0 0) 0x7f96a41a8960 con 0x7f96a410e110 2024-09-16T08:46:07.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96a41a9b60 con 0x7f96a410e110 2024-09-16T08:46:07.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2254683106 0 0) 0x7f96a41a7760 con 0x7f96a410a570 2024-09-16T08:46:07.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f96a41a8960 con 0x7f96a410a570 2024-09-16T08:46:07.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f9698001870 con 0x7f96a410a570 2024-09-16T08:46:07.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2386008012 0 0) 0x7f968c003160 con 0x7f96a4101a00 2024-09-16T08:46:07.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f96a41a7760 con 0x7f96a4101a00 2024-09-16T08:46:07.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f9694001270 con 0x7f96a4101a00 2024-09-16T08:46:07.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1293882298 0 0) 0x7f96a41a9b60 con 0x7f96a410e110 2024-09-16T08:46:07.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f968c003160 con 0x7f96a410e110 2024-09-16T08:46:07.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f969c001930 con 0x7f96a410e110 2024-09-16T08:46:07.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2395330044 0 0) 0x7f96a41a8960 con 0x7f96a410a570 2024-09-16T08:46:07.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 >> v1:172.21.15.169:6789/0 conn(0x7f96a410e110 legacy=0x7f96a41a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:07.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 >> v1:172.21.15.160:6789/0 conn(0x7f96a4101a00 legacy=0x7f96a410d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:07.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a41aae00 con 0x7f96a410a570 2024-09-16T08:46:07.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f96a9287640 1 -- 172.21.15.160:0/2193571659 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f96a41a9d90 con 0x7f96a410a570 2024-09-16T08:46:07.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f96a9287640 1 -- 172.21.15.160:0/2193571659 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f96a41aa370 con 0x7f96a410a570 2024-09-16T08:46:07.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.112+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f9698003a80 con 0x7f96a410a570 2024-09-16T08:46:07.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.113+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f9698005ac0 con 0x7f96a410a570 2024-09-16T08:46:07.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.113+0000 7f96a9287640 1 -- 172.21.15.160:0/2193571659 --> v1:172.21.15.160:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f9690000fb0 con 0x7f96a410a570 2024-09-16T08:46:07.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.114+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 21) ==== 103047+0+0 (unknown 1719902048 0 0) 0x7f9698003610 con 0x7f96a410a570 2024-09-16T08:46:07.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.115+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(64..64 src has 1..64) ==== 6172+0+0 (unknown 440296652 0 0) 0x7f9698099100 con 0x7f96a410a570 2024-09-16T08:46:07.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.115+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 --> v1:172.21.15.160:6801/520428698 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9694001d30 con 0x7f968c086540 2024-09-16T08:46:07.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.115+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (unknown 2884312405 0 0) 0x7f9698099490 con 0x7f96a410a570 2024-09-16T08:46:07.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.116+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== osd.0 v1:172.21.15.160:6801/520428698 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f9694001d30 con 0x7f968c086540 2024-09-16T08:46:07.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.130+0000 7f96a9287640 1 -- 172.21.15.160:0/2193571659 --> v1:172.21.15.160:6801/520428698 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9690002ca0 con 0x7f968c086540 2024-09-16T08:46:07.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.130+0000 7f968bfff640 1 -- 172.21.15.160:0/2193571659 <== osd.0 v1:172.21.15.160:6801/520428698 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 3385987527) 0x7f9690002ca0 con 0x7f968c086540 2024-09-16T08:46:07.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.130+0000 7f96a9287640 1 -- 172.21.15.160:0/2193571659 >> v1:172.21.15.160:6801/520428698 conn(0x7f968c086540 legacy=0x7f968c088980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:07.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.130+0000 7f96a9287640 1 -- 172.21.15.160:0/2193571659 >> v1:172.21.15.160:6800/2859204856 conn(0x7f968c07bcd0 legacy=0x7f968c07e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:07.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.130+0000 7f96a9287640 1 -- 172.21.15.160:0/2193571659 >> v1:172.21.15.160:6790/0 conn(0x7f96a410a570 legacy=0x7f96a41a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:07.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.130+0000 7f96a37fe640 1 -- 172.21.15.160:0/2193571659 reap_dead start 2024-09-16T08:46:07.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.131+0000 7f96a9287640 1 -- 172.21.15.160:0/2193571659 shutdown_connections 2024-09-16T08:46:07.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.131+0000 7f96a9287640 1 -- 172.21.15.160:0/2193571659 >> 172.21.15.160:0/2193571659 conn(0x7f96a40fd450 msgr2=0x7f96a4110da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:07.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.131+0000 7f96a9287640 1 -- 172.21.15.160:0/2193571659 shutdown_connections 2024-09-16T08:46:07.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:07.131+0000 7f96a9287640 1 -- 172.21.15.160:0/2193571659 wait complete. 2024-09-16T08:46:07.530 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:07.530 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:07.530 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:07.530 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:07.530 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:07.531 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:07.531 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:07.531 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:07.531 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:07.531 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:07.531 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:07.531 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:07.531 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:07.531 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:07.531 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:07.531 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:07.532 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:07.532 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:07 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:07.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:07 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:07.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:07 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:07.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:07 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:07.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:07 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:07.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:07 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:07.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:07 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:07.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:07 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:07.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:07 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:07.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:07 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:07.976 INFO:teuthology.orchestra.run.smithi160.stdout:34359738419 2024-09-16T08:46:07.976 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T08:46:08.403 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:46:08.430 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:46:08.431 INFO:teuthology.orchestra.run.smithi160.stdout:163208757267 2024-09-16T08:46:08.432 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T08:46:08.439 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:46:08.532 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:08 smithi160 ceph-mon[30137]: Adjusting osd_memory_target on smithi169 to 3760M 2024-09-16T08:46:08.532 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:08 smithi160 ceph-mon[30137]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-16T08:46:08.532 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:08 smithi160 ceph-mon[30137]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-16T08:46:08.532 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:08 smithi160 ceph-mon[40028]: Adjusting osd_memory_target on smithi169 to 3760M 2024-09-16T08:46:08.532 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:08 smithi160 ceph-mon[40028]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-16T08:46:08.532 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:08 smithi160 ceph-mon[40028]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-16T08:46:08.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:08 smithi169 ceph-mon[33465]: Adjusting osd_memory_target on smithi169 to 3760M 2024-09-16T08:46:08.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:08 smithi169 ceph-mon[33465]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-16T08:46:08.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:08 smithi169 ceph-mon[33465]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-16T08:46:08.691 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:46:09.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.119+0000 7fa810f1d640 1 -- 172.21.15.160:0/1059288188 >> v1:172.21.15.160:6790/0 conn(0x7fa80c10b8c0 legacy=0x7fa80c10dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:09.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.119+0000 7fa810f1d640 1 -- 172.21.15.160:0/1059288188 shutdown_connections 2024-09-16T08:46:09.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.119+0000 7fa810f1d640 1 -- 172.21.15.160:0/1059288188 >> 172.21.15.160:0/1059288188 conn(0x7fa80c0fd490 msgr2=0x7fa80c0ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:09.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.120+0000 7fa810f1d640 1 -- 172.21.15.160:0/1059288188 shutdown_connections 2024-09-16T08:46:09.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.120+0000 7fa810f1d640 1 -- 172.21.15.160:0/1059288188 wait complete. 2024-09-16T08:46:09.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.120+0000 7fa810f1d640 1 Processor -- start 2024-09-16T08:46:09.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.120+0000 7fa810f1d640 1 -- start start 2024-09-16T08:46:09.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa810f1d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa80c1a77c0 con 0x7fa80c1038e0 2024-09-16T08:46:09.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa810f1d640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa80c1a89c0 con 0x7fa80c10f3d0 2024-09-16T08:46:09.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa810f1d640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa80c1a9bc0 con 0x7fa80c10b8c0 2024-09-16T08:46:09.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa80a59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fa80c1038e0 0x7fa80c10af80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:59200/0 (socket says 172.21.15.160:59200) 2024-09-16T08:46:09.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa80a59c640 1 -- 172.21.15.160:0/4156854287 learned_addr learned my addr 172.21.15.160:0/4156854287 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:09.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2434581395 0 0) 0x7fa80c1a77c0 con 0x7fa80c1038e0 2024-09-16T08:46:09.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7f8003520 con 0x7fa80c1038e0 2024-09-16T08:46:09.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4203984116 0 0) 0x7fa80c1a9bc0 con 0x7fa80c10b8c0 2024-09-16T08:46:09.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa80c1a77c0 con 0x7fa80c10b8c0 2024-09-16T08:46:09.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1011455759 0 0) 0x7fa80c1a89c0 con 0x7fa80c10f3d0 2024-09-16T08:46:09.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa80c1a9bc0 con 0x7fa80c10f3d0 2024-09-16T08:46:09.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3650827366 0 0) 0x7fa7f8003520 con 0x7fa80c1038e0 2024-09-16T08:46:09.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa80c1a89c0 con 0x7fa80c1038e0 2024-09-16T08:46:09.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fa7fc001390 con 0x7fa80c1038e0 2024-09-16T08:46:09.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3942101673 0 0) 0x7fa80c1a77c0 con 0x7fa80c10b8c0 2024-09-16T08:46:09.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa7f8003520 con 0x7fa80c10b8c0 2024-09-16T08:46:09.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fa7f4001580 con 0x7fa80c10b8c0 2024-09-16T08:46:09.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1221741446 0 0) 0x7fa80c1a9bc0 con 0x7fa80c10f3d0 2024-09-16T08:46:09.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa80c1a77c0 con 0x7fa80c10f3d0 2024-09-16T08:46:09.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fa804001c70 con 0x7fa80c10f3d0 2024-09-16T08:46:09.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1908951741 0 0) 0x7fa80c1a89c0 con 0x7fa80c1038e0 2024-09-16T08:46:09.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 >> v1:172.21.15.160:6790/0 conn(0x7fa80c10b8c0 legacy=0x7fa80c1a2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:09.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 >> v1:172.21.15.169:6789/0 conn(0x7fa80c10f3d0 legacy=0x7fa80c1a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:09.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c1aae60 con 0x7fa80c1038e0 2024-09-16T08:46:09.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa810f1d640 1 -- 172.21.15.160:0/4156854287 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa80c1a9df0 con 0x7fa80c1038e0 2024-09-16T08:46:09.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.121+0000 7fa810f1d640 1 -- 172.21.15.160:0/4156854287 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa80c1aa3d0 con 0x7fa80c1038e0 2024-09-16T08:46:09.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.122+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fa7fc003a60 con 0x7fa80c1038e0 2024-09-16T08:46:09.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.122+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fa7fc005be0 con 0x7fa80c1038e0 2024-09-16T08:46:09.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.122+0000 7fa810f1d640 1 -- 172.21.15.160:0/4156854287 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7ec0050b0 con 0x7fa80c1038e0 2024-09-16T08:46:09.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.123+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7fa7fc0035f0 con 0x7fa80c1038e0 2024-09-16T08:46:09.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.124+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(64..64 src has 1..64) ==== 6172+0+0 (unknown 440296652 0 0) 0x7fa7fc0990e0 con 0x7fa80c1038e0 2024-09-16T08:46:09.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.128+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fa7fc0646f0 con 0x7fa80c1038e0 2024-09-16T08:46:09.154 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:46:09.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.333+0000 7fa810f1d640 1 -- 172.21.15.160:0/4156854287 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa7ec0053a0 con 0x7fa80c1038e0 2024-09-16T08:46:09.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.333+0000 7fa8037fe640 1 -- 172.21.15.160:0/4156854287 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (unknown 832126871 0 50725365) 0x7fa7fc069630 con 0x7fa80c1038e0 2024-09-16T08:46:09.334 INFO:teuthology.orchestra.run.smithi160.stdout:55834574893 2024-09-16T08:46:09.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.336+0000 7fa810f1d640 1 -- 172.21.15.160:0/4156854287 >> v1:172.21.15.160:6800/2859204856 conn(0x7fa7f807bd50 legacy=0x7fa7f807e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:09.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.336+0000 7fa810f1d640 1 -- 172.21.15.160:0/4156854287 >> v1:172.21.15.160:6789/0 conn(0x7fa80c1038e0 legacy=0x7fa80c10af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:09.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.336+0000 7fa810f1d640 1 -- 172.21.15.160:0/4156854287 shutdown_connections 2024-09-16T08:46:09.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.336+0000 7fa810f1d640 1 -- 172.21.15.160:0/4156854287 >> 172.21.15.160:0/4156854287 conn(0x7fa80c0fd490 msgr2=0x7fa80c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:09.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.336+0000 7fa810f1d640 1 -- 172.21.15.160:0/4156854287 shutdown_connections 2024-09-16T08:46:09.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.336+0000 7fa810f1d640 1 -- 172.21.15.160:0/4156854287 wait complete. 2024-09-16T08:46:09.950 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:46:09.950 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:46:09.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.959+0000 7f11f3b54640 1 -- 172.21.15.160:0/2426971211 >> v1:172.21.15.160:6790/0 conn(0x7f11ec10e180 legacy=0x7f11ec110620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:09.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.959+0000 7f11f3b54640 1 -- 172.21.15.160:0/2426971211 shutdown_connections 2024-09-16T08:46:09.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.959+0000 7f11f3b54640 1 -- 172.21.15.160:0/2426971211 >> 172.21.15.160:0/2426971211 conn(0x7f11ec0780f0 msgr2=0x7f11ec078500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:09.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.959+0000 7f11f3b54640 1 -- 172.21.15.160:0/2426971211 shutdown_connections 2024-09-16T08:46:09.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.960+0000 7f11f3b54640 1 -- 172.21.15.160:0/2426971211 wait complete. 2024-09-16T08:46:09.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.960+0000 7f11f3b54640 1 Processor -- start 2024-09-16T08:46:09.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.960+0000 7f11f3b54640 1 -- start start 2024-09-16T08:46:09.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.960+0000 7f11f3b54640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f11ec1a5630 con 0x7f11ec10a4f0 2024-09-16T08:46:09.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.960+0000 7f11f3b54640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f11ec1a6830 con 0x7f11ec10e180 2024-09-16T08:46:09.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.960+0000 7f11f3b54640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f11ec1a7a30 con 0x7f11ec105590 2024-09-16T08:46:09.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.961+0000 7f11f10ef640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f11ec10a4f0 0x7f11ec1a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:59216/0 (socket says 172.21.15.160:59216) 2024-09-16T08:46:09.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.961+0000 7f11f10ef640 1 -- 172.21.15.160:0/2865847696 learned_addr learned my addr 172.21.15.160:0/2865847696 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:09.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.961+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2634402142 0 0) 0x7f11ec1a5630 con 0x7f11ec10a4f0 2024-09-16T08:46:09.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.961+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11d8003520 con 0x7f11ec10a4f0 2024-09-16T08:46:09.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.961+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 968452048 0 0) 0x7f11ec1a6830 con 0x7f11ec10e180 2024-09-16T08:46:09.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.961+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11ec1a5630 con 0x7f11ec10e180 2024-09-16T08:46:09.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.961+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1849140449 0 0) 0x7f11ec1a7a30 con 0x7f11ec105590 2024-09-16T08:46:09.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.961+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11ec1a6830 con 0x7f11ec105590 2024-09-16T08:46:09.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.962+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 945973032 0 0) 0x7f11d8003520 con 0x7f11ec10a4f0 2024-09-16T08:46:09.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.962+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f11ec1a7a30 con 0x7f11ec10a4f0 2024-09-16T08:46:09.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.962+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f11dc001990 con 0x7f11ec10a4f0 2024-09-16T08:46:09.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.962+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 839613751 0 0) 0x7f11ec1a5630 con 0x7f11ec10e180 2024-09-16T08:46:09.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.962+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f11d8003520 con 0x7f11ec10e180 2024-09-16T08:46:09.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.962+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f11e4004770 con 0x7f11ec10e180 2024-09-16T08:46:09.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.962+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2258641125 0 0) 0x7f11ec1a6830 con 0x7f11ec105590 2024-09-16T08:46:09.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.962+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f11ec1a5630 con 0x7f11ec105590 2024-09-16T08:46:09.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.962+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f11e80013b0 con 0x7f11ec105590 2024-09-16T08:46:09.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.962+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 <== mon.1 v1:172.21.15.169:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3733501249 0 0) 0x7f11d8003520 con 0x7f11ec10e180 2024-09-16T08:46:09.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.962+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 >> v1:172.21.15.160:6790/0 conn(0x7f11ec105590 legacy=0x7f11ec104970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:09.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.962+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 >> v1:172.21.15.160:6789/0 conn(0x7f11ec10a4f0 legacy=0x7f11ec1a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:09.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.962+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ec1a8cd0 con 0x7f11ec10e180 2024-09-16T08:46:09.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.962+0000 7f11f3b54640 1 -- 172.21.15.160:0/2865847696 --> v1:172.21.15.169:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f11ec1a6a60 con 0x7f11ec10e180 2024-09-16T08:46:09.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.962+0000 7f11f3b54640 1 -- 172.21.15.160:0/2865847696 --> v1:172.21.15.169:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f11ec1a6fc0 con 0x7f11ec10e180 2024-09-16T08:46:09.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.963+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 <== mon.1 v1:172.21.15.169:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f11e4003c70 con 0x7f11ec10e180 2024-09-16T08:46:09.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.963+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 <== mon.1 v1:172.21.15.169:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f11e4005070 con 0x7f11ec10e180 2024-09-16T08:46:09.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.965+0000 7f11f3b54640 1 -- 172.21.15.160:0/2865847696 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d0005080 con 0x7f11ec10e180 2024-09-16T08:46:09.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.965+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 <== mon.1 v1:172.21.15.169:6789/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7f11e40052b0 con 0x7f11ec10e180 2024-09-16T08:46:09.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.966+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 <== mon.1 v1:172.21.15.169:6789/0 8 ==== osd_map(64..64 src has 1..64) ==== 6172+0+0 (unknown 440296652 0 0) 0x7f11e40982b0 con 0x7f11ec10e180 2024-09-16T08:46:09.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:09.970+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 <== mon.1 v1:172.21.15.169:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f11e40638c0 con 0x7f11ec10e180 2024-09-16T08:46:10.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.149+0000 7f11f3b54640 1 -- 172.21.15.160:0/2865847696 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f11d0005370 con 0x7f11ec10e180 2024-09-16T08:46:10.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.149+0000 7f11e2ffd640 1 -- 172.21.15.160:0/2865847696 <== mon.1 v1:172.21.15.169:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (unknown 92182286 0 3693870858) 0x7f11e4068800 con 0x7f11ec10e180 2024-09-16T08:46:10.150 INFO:teuthology.orchestra.run.smithi160.stdout:77309411369 2024-09-16T08:46:10.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.152+0000 7f11f3b54640 1 -- 172.21.15.160:0/2865847696 >> v1:172.21.15.160:6800/2859204856 conn(0x7f11d807df90 legacy=0x7f11d8080450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.152+0000 7f11f3b54640 1 -- 172.21.15.160:0/2865847696 >> v1:172.21.15.169:6789/0 conn(0x7f11ec10e180 legacy=0x7f11ec1a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.152+0000 7f11f3b54640 1 -- 172.21.15.160:0/2865847696 shutdown_connections 2024-09-16T08:46:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.152+0000 7f11f3b54640 1 -- 172.21.15.160:0/2865847696 >> 172.21.15.160:0/2865847696 conn(0x7f11ec0780f0 msgr2=0x7f11ec107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.152+0000 7f11f3b54640 1 -- 172.21.15.160:0/2865847696 shutdown_connections 2024-09-16T08:46:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.152+0000 7f11f3b54640 1 -- 172.21.15.160:0/2865847696 wait complete. 2024-09-16T08:46:10.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:10 smithi160 ceph-mon[30137]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-16T08:46:10.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4156854287' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-16T08:46:10.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2865847696' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-16T08:46:10.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:10 smithi160 ceph-mon[40028]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-16T08:46:10.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4156854287' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-16T08:46:10.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2865847696' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-16T08:46:10.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:10 smithi169 ceph-mon[33465]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-16T08:46:10.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4156854287' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-16T08:46:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2865847696' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-16T08:46:11.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.823+0000 7fb17979f640 1 -- 172.21.15.160:0/4173876908 >> v1:172.21.15.160:6790/0 conn(0x7fb1741038e0 legacy=0x7fb174103cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:11.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.823+0000 7fb17979f640 1 -- 172.21.15.160:0/4173876908 shutdown_connections 2024-09-16T08:46:11.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.823+0000 7fb17979f640 1 -- 172.21.15.160:0/4173876908 >> 172.21.15.160:0/4173876908 conn(0x7fb1740fd490 msgr2=0x7fb1740ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:11.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.823+0000 7fb17979f640 1 -- 172.21.15.160:0/4173876908 shutdown_connections 2024-09-16T08:46:11.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.823+0000 7fb17979f640 1 -- 172.21.15.160:0/4173876908 wait complete. 2024-09-16T08:46:11.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.823+0000 7fb17979f640 1 Processor -- start 2024-09-16T08:46:11.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.824+0000 7fb17979f640 1 -- start start 2024-09-16T08:46:11.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.824+0000 7fb17979f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb1741a7600 con 0x7fb17410b8c0 2024-09-16T08:46:11.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.824+0000 7fb17979f640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb1741a87e0 con 0x7fb1741038e0 2024-09-16T08:46:11.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.824+0000 7fb17979f640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb1741a99e0 con 0x7fb17410f3d0 2024-09-16T08:46:11.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.824+0000 7fb1737fe640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7fb17410f3d0 0x7fb1741a5dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:55656/0 (socket says 172.21.15.160:55656) 2024-09-16T08:46:11.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.824+0000 7fb1737fe640 1 -- 172.21.15.160:0/4241935351 learned_addr learned my addr 172.21.15.160:0/4241935351 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:11.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.824+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1129002883 0 0) 0x7fb1741a7600 con 0x7fb17410b8c0 2024-09-16T08:46:11.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.824+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb160003520 con 0x7fb17410b8c0 2024-09-16T08:46:11.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.824+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2622746087 0 0) 0x7fb1741a99e0 con 0x7fb17410f3d0 2024-09-16T08:46:11.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.824+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1741a7600 con 0x7fb17410f3d0 2024-09-16T08:46:11.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.824+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2639027496 0 0) 0x7fb1741a87e0 con 0x7fb1741038e0 2024-09-16T08:46:11.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.824+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1741a99e0 con 0x7fb1741038e0 2024-09-16T08:46:11.137 INFO:teuthology.orchestra.run.smithi160.stdout:163208757269 2024-09-16T08:46:11.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3512457772 0 0) 0x7fb160003520 con 0x7fb17410b8c0 2024-09-16T08:46:11.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb1741a87e0 con 0x7fb17410b8c0 2024-09-16T08:46:11.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb15c0013b0 con 0x7fb17410b8c0 2024-09-16T08:46:11.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 393829095 0 0) 0x7fb1741a7600 con 0x7fb17410f3d0 2024-09-16T08:46:11.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb160003520 con 0x7fb17410f3d0 2024-09-16T08:46:11.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb16c001cc0 con 0x7fb17410f3d0 2024-09-16T08:46:11.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2925552325 0 0) 0x7fb1741a99e0 con 0x7fb1741038e0 2024-09-16T08:46:11.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb1741a7600 con 0x7fb1741038e0 2024-09-16T08:46:11.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb164001500 con 0x7fb1741038e0 2024-09-16T08:46:11.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3765719656 0 0) 0x7fb1741a87e0 con 0x7fb17410b8c0 2024-09-16T08:46:11.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 >> v1:172.21.15.160:6790/0 conn(0x7fb17410f3d0 legacy=0x7fb1741a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:11.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 >> v1:172.21.15.169:6789/0 conn(0x7fb1741038e0 legacy=0x7fb17410ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:11.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1741aac80 con 0x7fb17410b8c0 2024-09-16T08:46:11.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb17979f640 1 -- 172.21.15.160:0/4241935351 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb1741a9c10 con 0x7fb17410b8c0 2024-09-16T08:46:11.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb17979f640 1 -- 172.21.15.160:0/4241935351 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb1741aa0c0 con 0x7fb17410b8c0 2024-09-16T08:46:11.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fb15c0015d0 con 0x7fb17410b8c0 2024-09-16T08:46:11.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.825+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb15c004a40 con 0x7fb17410b8c0 2024-09-16T08:46:11.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.826+0000 7fb17979f640 1 -- 172.21.15.160:0/4241935351 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb17406ba60 con 0x7fb17410b8c0 2024-09-16T08:46:11.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.827+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7fb15c0038a0 con 0x7fb17410b8c0 2024-09-16T08:46:11.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.827+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(64..64 src has 1..64) ==== 6172+0+0 (unknown 440296652 0 0) 0x7fb15c098090 con 0x7fb17410b8c0 2024-09-16T08:46:11.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:10.831+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fb15c0636a0 con 0x7fb17410b8c0 2024-09-16T08:46:11.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:11.013+0000 7fb17979f640 1 -- 172.21.15.160:0/4241935351 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fb1741150c0 con 0x7fb17410b8c0 2024-09-16T08:46:11.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:11.013+0000 7fb170ff9640 1 -- 172.21.15.160:0/4241935351 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (unknown 2131975755 0 1259768193) 0x7fb15c0685e0 con 0x7fb17410b8c0 2024-09-16T08:46:11.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:11.015+0000 7fb17979f640 1 -- 172.21.15.160:0/4241935351 >> v1:172.21.15.160:6800/2859204856 conn(0x7fb16007bda0 legacy=0x7fb16007e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:11.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:11.015+0000 7fb17979f640 1 -- 172.21.15.160:0/4241935351 >> v1:172.21.15.160:6789/0 conn(0x7fb17410b8c0 legacy=0x7fb1741a2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:11.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:11.015+0000 7fb17979f640 1 -- 172.21.15.160:0/4241935351 shutdown_connections 2024-09-16T08:46:11.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:11.015+0000 7fb17979f640 1 -- 172.21.15.160:0/4241935351 >> 172.21.15.160:0/4241935351 conn(0x7fb1740fd490 msgr2=0x7fb1740fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:11.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:11.015+0000 7fb17979f640 1 -- 172.21.15.160:0/4241935351 shutdown_connections 2024-09-16T08:46:11.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:11.016+0000 7fb17979f640 1 -- 172.21.15.160:0/4241935351 wait complete. 2024-09-16T08:46:11.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4241935351' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-16T08:46:11.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4241935351' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-16T08:46:11.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4241935351' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-16T08:46:11.893 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757267 got 163208757269 for osd.5 2024-09-16T08:46:11.894 DEBUG:teuthology.parallel:result is None 2024-09-16T08:46:12.384 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:12 smithi160 ceph-mon[30137]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T08:46:12.384 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:12 smithi160 ceph-mon[40028]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T08:46:12.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.389+0000 7f340a83c640 1 -- 172.21.15.160:0/2339283306 >> v1:172.21.15.160:6790/0 conn(0x7f3404101a20 legacy=0x7f3404101e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:12.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.390+0000 7f340a83c640 1 -- 172.21.15.160:0/2339283306 shutdown_connections 2024-09-16T08:46:12.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.390+0000 7f340a83c640 1 -- 172.21.15.160:0/2339283306 >> 172.21.15.160:0/2339283306 conn(0x7f34040fd4b0 msgr2=0x7f34040ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:12.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.390+0000 7f340a83c640 1 -- 172.21.15.160:0/2339283306 shutdown_connections 2024-09-16T08:46:12.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.390+0000 7f340a83c640 1 -- 172.21.15.160:0/2339283306 wait complete. 2024-09-16T08:46:12.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.390+0000 7f340a83c640 1 Processor -- start 2024-09-16T08:46:12.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.390+0000 7f340a83c640 1 -- start start 2024-09-16T08:46:12.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f340a83c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f34041a65f0 con 0x7f340410e130 2024-09-16T08:46:12.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f340a83c640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f34041a77f0 con 0x7f340410a590 2024-09-16T08:46:12.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f340a83c640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f34041a89f0 con 0x7f3404101a20 2024-09-16T08:46:12.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f3403fff640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f3404101a20 0x7f340410d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:55672/0 (socket says 172.21.15.160:55672) 2024-09-16T08:46:12.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f3403fff640 1 -- 172.21.15.160:0/3965017712 learned_addr learned my addr 172.21.15.160:0/3965017712 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:12.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3916498296 0 0) 0x7f34041a65f0 con 0x7f340410e130 2024-09-16T08:46:12.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33ec003160 con 0x7f340410e130 2024-09-16T08:46:12.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2678920403 0 0) 0x7f34041a89f0 con 0x7f3404101a20 2024-09-16T08:46:12.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34041a65f0 con 0x7f3404101a20 2024-09-16T08:46:12.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4268620072 0 0) 0x7f34041a77f0 con 0x7f340410a590 2024-09-16T08:46:12.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34041a89f0 con 0x7f340410a590 2024-09-16T08:46:12.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1182431316 0 0) 0x7f33ec003160 con 0x7f340410e130 2024-09-16T08:46:12.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f34041a77f0 con 0x7f340410e130 2024-09-16T08:46:12.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f33fc001bc0 con 0x7f340410e130 2024-09-16T08:46:12.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3356377483 0 0) 0x7f34041a65f0 con 0x7f3404101a20 2024-09-16T08:46:12.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f33ec003160 con 0x7f3404101a20 2024-09-16T08:46:12.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f33f4001490 con 0x7f3404101a20 2024-09-16T08:46:12.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2399895059 0 0) 0x7f34041a89f0 con 0x7f340410a590 2024-09-16T08:46:12.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f34041a65f0 con 0x7f340410a590 2024-09-16T08:46:12.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4255092673 0 0) 0x7f34041a77f0 con 0x7f340410e130 2024-09-16T08:46:12.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 >> v1:172.21.15.160:6790/0 conn(0x7f3404101a20 legacy=0x7f340410d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:12.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 >> v1:172.21.15.169:6789/0 conn(0x7f340410a590 legacy=0x7f34041a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:12.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34041a9c90 con 0x7f340410e130 2024-09-16T08:46:12.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.391+0000 7f340a83c640 1 -- 172.21.15.160:0/3965017712 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f34041a8c20 con 0x7f340410e130 2024-09-16T08:46:12.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.392+0000 7f340a83c640 1 -- 172.21.15.160:0/3965017712 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f34041a91d0 con 0x7f340410e130 2024-09-16T08:46:12.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.392+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f33fc003de0 con 0x7f340410e130 2024-09-16T08:46:12.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.392+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f33fc004cb0 con 0x7f340410e130 2024-09-16T08:46:12.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.393+0000 7f340a83c640 1 -- 172.21.15.160:0/3965017712 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3404102cd0 con 0x7f340410e130 2024-09-16T08:46:12.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.393+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7f33fc003970 con 0x7f340410e130 2024-09-16T08:46:12.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.394+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(64..64 src has 1..64) ==== 6172+0+0 (unknown 440296652 0 0) 0x7f33fc098430 con 0x7f340410e130 2024-09-16T08:46:12.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.398+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f33fc004c90 con 0x7f340410e130 2024-09-16T08:46:12.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.583+0000 7f5734a64640 1 -- 172.21.15.160:0/65318577 >> v1:172.21.15.160:6789/0 conn(0x7f57301038e0 legacy=0x7f5730103cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:12.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.583+0000 7f5734a64640 1 -- 172.21.15.160:0/65318577 shutdown_connections 2024-09-16T08:46:12.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.583+0000 7f5734a64640 1 -- 172.21.15.160:0/65318577 >> 172.21.15.160:0/65318577 conn(0x7f57300fd450 msgr2=0x7f57300ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:12.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.583+0000 7f5734a64640 1 -- 172.21.15.160:0/65318577 shutdown_connections 2024-09-16T08:46:12.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.583+0000 7f5734a64640 1 -- 172.21.15.160:0/65318577 wait complete. 2024-09-16T08:46:12.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.584+0000 7f5734a64640 1 Processor -- start 2024-09-16T08:46:12.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.584+0000 7f5734a64640 1 -- start start 2024-09-16T08:46:12.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.584+0000 7f5734a64640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f5730112a00 con 0x7f573010f3d0 2024-09-16T08:46:12.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.584+0000 7f5734a64640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f57301aacb0 con 0x7f57301038e0 2024-09-16T08:46:12.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.584+0000 7f5734a64640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f57301abe90 con 0x7f573010b8c0 2024-09-16T08:46:12.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.584+0000 7f572ed9d640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f573010f3d0 0x7f57301a8580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:59280/0 (socket says 172.21.15.160:59280) 2024-09-16T08:46:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.584+0000 7f572ed9d640 1 -- 172.21.15.160:0/1377351951 learned_addr learned my addr 172.21.15.160:0/1377351951 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.584+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2082739412 0 0) 0x7f57301abe90 con 0x7f573010b8c0 2024-09-16T08:46:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.584+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f571c003160 con 0x7f573010b8c0 2024-09-16T08:46:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.584+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4192679708 0 0) 0x7f5730112a00 con 0x7f573010f3d0 2024-09-16T08:46:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.584+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57301abe90 con 0x7f573010f3d0 2024-09-16T08:46:12.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.585+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 553377720 0 0) 0x7f57301aacb0 con 0x7f57301038e0 2024-09-16T08:46:12.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.585+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5730112a00 con 0x7f57301038e0 2024-09-16T08:46:12.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.585+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1745986301 0 0) 0x7f571c003160 con 0x7f573010b8c0 2024-09-16T08:46:12.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.585+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f57301aacb0 con 0x7f573010b8c0 2024-09-16T08:46:12.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.585+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 753729026 0 0) 0x7f57301abe90 con 0x7f573010f3d0 2024-09-16T08:46:12.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.585+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f571c003160 con 0x7f573010f3d0 2024-09-16T08:46:12.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.585+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f57180019e0 con 0x7f573010b8c0 2024-09-16T08:46:12.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.585+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f5728001c70 con 0x7f573010f3d0 2024-09-16T08:46:12.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.585+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 414338590 0 0) 0x7f57301aacb0 con 0x7f573010b8c0 2024-09-16T08:46:12.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.585+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 >> v1:172.21.15.169:6789/0 conn(0x7f57301038e0 legacy=0x7f5730111de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:12.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.585+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 >> v1:172.21.15.160:6789/0 conn(0x7f573010f3d0 legacy=0x7f57301a8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:12.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.585+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57301ad150 con 0x7f573010b8c0 2024-09-16T08:46:12.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.585+0000 7f5734a64640 1 -- 172.21.15.160:0/1377351951 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f57301ac0c0 con 0x7f573010b8c0 2024-09-16T08:46:12.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.585+0000 7f5734a64640 1 -- 172.21.15.160:0/1377351951 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f57301ac6f0 con 0x7f573010b8c0 2024-09-16T08:46:12.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.585+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f5718001b80 con 0x7f573010b8c0 2024-09-16T08:46:12.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.585+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f5718004990 con 0x7f573010b8c0 2024-09-16T08:46:12.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.586+0000 7f5734a64640 1 -- 172.21.15.160:0/1377351951 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57100050b0 con 0x7f573010b8c0 2024-09-16T08:46:12.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.587+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7f5718005c60 con 0x7f573010b8c0 2024-09-16T08:46:12.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.588+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(64..64 src has 1..64) ==== 6172+0+0 (unknown 440296652 0 0) 0x7f5718098140 con 0x7f573010b8c0 2024-09-16T08:46:12.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.588+0000 7f340a83c640 1 -- 172.21.15.160:0/3965017712 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3404113ee0 con 0x7f340410e130 2024-09-16T08:46:12.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.588+0000 7f34017fa640 1 -- 172.21.15.160:0/3965017712 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (unknown 1274345170 0 1536791651) 0x7f33fc063a40 con 0x7f340410e130 2024-09-16T08:46:12.595 INFO:teuthology.orchestra.run.smithi160.stdout:188978561039 2024-09-16T08:46:12.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.590+0000 7f340a83c640 1 -- 172.21.15.160:0/3965017712 >> v1:172.21.15.160:6800/2859204856 conn(0x7f33ec07df20 legacy=0x7f33ec0803e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:12.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.590+0000 7f340a83c640 1 -- 172.21.15.160:0/3965017712 >> v1:172.21.15.160:6789/0 conn(0x7f340410e130 legacy=0x7f34041a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:12.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.591+0000 7f340a83c640 1 -- 172.21.15.160:0/3965017712 shutdown_connections 2024-09-16T08:46:12.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.591+0000 7f340a83c640 1 -- 172.21.15.160:0/3965017712 >> 172.21.15.160:0/3965017712 conn(0x7f34040fd4b0 msgr2=0x7f3404110da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:12.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.591+0000 7f340a83c640 1 -- 172.21.15.160:0/3965017712 shutdown_connections 2024-09-16T08:46:12.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.591+0000 7f340a83c640 1 -- 172.21.15.160:0/3965017712 wait complete. 2024-09-16T08:46:12.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.592+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f5718063750 con 0x7f573010b8c0 2024-09-16T08:46:12.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:12 smithi169 ceph-mon[33465]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T08:46:12.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.777+0000 7f5734a64640 1 -- 172.21.15.160:0/1377351951 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f57100053a0 con 0x7f573010b8c0 2024-09-16T08:46:12.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.778+0000 7f57277fe640 1 -- 172.21.15.160:0/1377351951 <== mon.2 v1:172.21.15.160:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (unknown 1823589948 0 3457586501) 0x7f5718068690 con 0x7f573010b8c0 2024-09-16T08:46:12.778 INFO:teuthology.orchestra.run.smithi160.stdout:137438953501 2024-09-16T08:46:12.780 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.780+0000 7f5734a64640 1 -- 172.21.15.160:0/1377351951 >> v1:172.21.15.160:6800/2859204856 conn(0x7f571c07ba40 legacy=0x7f571c07df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:12.780 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.780+0000 7f5734a64640 1 -- 172.21.15.160:0/1377351951 >> v1:172.21.15.160:6790/0 conn(0x7f573010b8c0 legacy=0x7f57301a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:12.780 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.780+0000 7f5734a64640 1 -- 172.21.15.160:0/1377351951 shutdown_connections 2024-09-16T08:46:12.780 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.780+0000 7f5734a64640 1 -- 172.21.15.160:0/1377351951 >> 172.21.15.160:0/1377351951 conn(0x7f57300fd450 msgr2=0x7f57300fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:12.780 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.780+0000 7f5734a64640 1 -- 172.21.15.160:0/1377351951 shutdown_connections 2024-09-16T08:46:12.781 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:12.780+0000 7f5734a64640 1 -- 172.21.15.160:0/1377351951 wait complete. 2024-09-16T08:46:13.259 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561037 got 188978561039 for osd.6 2024-09-16T08:46:13.259 DEBUG:teuthology.parallel:result is None 2024-09-16T08:46:13.498 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3965017712' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-16T08:46:13.498 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1377351951' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-16T08:46:13.499 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3965017712' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-16T08:46:13.499 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1377351951' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-16T08:46:13.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3965017712' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-16T08:46:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1377351951' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-16T08:46:13.863 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574891 got 55834574893 for osd.1 2024-09-16T08:46:13.863 DEBUG:teuthology.parallel:result is None 2024-09-16T08:46:14.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.030+0000 7fc700075640 1 -- 172.21.15.160:0/2094856162 >> v1:172.21.15.160:6790/0 conn(0x7fc6f8101a30 legacy=0x7fc6f8101e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:14.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.030+0000 7fc700075640 1 -- 172.21.15.160:0/2094856162 shutdown_connections 2024-09-16T08:46:14.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.030+0000 7fc700075640 1 -- 172.21.15.160:0/2094856162 >> 172.21.15.160:0/2094856162 conn(0x7fc6f80fd480 msgr2=0x7fc6f80ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:14.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.030+0000 7fc700075640 1 -- 172.21.15.160:0/2094856162 shutdown_connections 2024-09-16T08:46:14.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.030+0000 7fc700075640 1 -- 172.21.15.160:0/2094856162 wait complete. 2024-09-16T08:46:14.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.031+0000 7fc700075640 1 Processor -- start 2024-09-16T08:46:14.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.031+0000 7fc700075640 1 -- start start 2024-09-16T08:46:14.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.031+0000 7fc700075640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc6f81a77e0 con 0x7fc6f8101a30 2024-09-16T08:46:14.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.031+0000 7fc700075640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc6f81a89e0 con 0x7fc6f810a5a0 2024-09-16T08:46:14.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.031+0000 7fc700075640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc6f81a9be0 con 0x7fc6f810e140 2024-09-16T08:46:14.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.031+0000 7fc6fde11640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fc6f8101a30 0x7fc6f806bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:59298/0 (socket says 172.21.15.160:59298) 2024-09-16T08:46:14.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.031+0000 7fc6fde11640 1 -- 172.21.15.160:0/2090456708 learned_addr learned my addr 172.21.15.160:0/2090456708 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:14.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.031+0000 7fc6fe612640 1 --1- 172.21.15.160:0/2090456708 >> v1:172.21.15.160:6790/0 conn(0x7fc6f810e140 0x7fc6f81a5ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:55710/0 (socket says 172.21.15.160:55710) 2024-09-16T08:46:14.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.031+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 997746390 0 0) 0x7fc6f81a77e0 con 0x7fc6f8101a30 2024-09-16T08:46:14.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.031+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6e4003160 con 0x7fc6f8101a30 2024-09-16T08:46:14.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.031+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1045885527 0 0) 0x7fc6f81a89e0 con 0x7fc6f810a5a0 2024-09-16T08:46:14.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.031+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6f81a77e0 con 0x7fc6f810a5a0 2024-09-16T08:46:14.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.032+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3231908203 0 0) 0x7fc6f81a77e0 con 0x7fc6f810a5a0 2024-09-16T08:46:14.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.032+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc6f81a89e0 con 0x7fc6f810a5a0 2024-09-16T08:46:14.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.032+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fc6e80018f0 con 0x7fc6f810a5a0 2024-09-16T08:46:14.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.032+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 323955716 0 0) 0x7fc6f81a9be0 con 0x7fc6f810e140 2024-09-16T08:46:14.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.032+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6f81a77e0 con 0x7fc6f810e140 2024-09-16T08:46:14.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.032+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1013288276 0 0) 0x7fc6e4003160 con 0x7fc6f8101a30 2024-09-16T08:46:14.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.032+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc6f81a9be0 con 0x7fc6f8101a30 2024-09-16T08:46:14.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.032+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fc6f4001530 con 0x7fc6f8101a30 2024-09-16T08:46:14.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.032+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 <== mon.1 v1:172.21.15.169:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3735741099 0 0) 0x7fc6f81a89e0 con 0x7fc6f810a5a0 2024-09-16T08:46:14.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.032+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 >> v1:172.21.15.160:6790/0 conn(0x7fc6f810e140 legacy=0x7fc6f81a5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:14.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.032+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 >> v1:172.21.15.160:6789/0 conn(0x7fc6f8101a30 legacy=0x7fc6f806bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:14.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.032+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f81aae80 con 0x7fc6f810a5a0 2024-09-16T08:46:14.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.032+0000 7fc700075640 1 -- 172.21.15.160:0/2090456708 --> v1:172.21.15.169:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f81a8c10 con 0x7fc6f810a5a0 2024-09-16T08:46:14.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.032+0000 7fc700075640 1 -- 172.21.15.160:0/2090456708 --> v1:172.21.15.169:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc6f81a9130 con 0x7fc6f810a5a0 2024-09-16T08:46:14.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.032+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 <== mon.1 v1:172.21.15.169:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fc6e8003a80 con 0x7fc6f810a5a0 2024-09-16T08:46:14.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.032+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 <== mon.1 v1:172.21.15.169:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fc6e80049c0 con 0x7fc6f810a5a0 2024-09-16T08:46:14.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.033+0000 7fc700075640 1 -- 172.21.15.160:0/2090456708 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6dc0050b0 con 0x7fc6f810a5a0 2024-09-16T08:46:14.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.034+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 <== mon.1 v1:172.21.15.169:6789/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7fc6e8003610 con 0x7fc6f810a5a0 2024-09-16T08:46:14.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.035+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 <== mon.1 v1:172.21.15.169:6789/0 8 ==== osd_map(64..64 src has 1..64) ==== 6172+0+0 (unknown 440296652 0 0) 0x7fc6e80980e0 con 0x7fc6f810a5a0 2024-09-16T08:46:14.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.039+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 <== mon.1 v1:172.21.15.169:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fc6e80636f0 con 0x7fc6f810a5a0 2024-09-16T08:46:14.221 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.221+0000 7fc700075640 1 -- 172.21.15.160:0/2090456708 --> v1:172.21.15.169:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc6dc0053a0 con 0x7fc6f810a5a0 2024-09-16T08:46:14.221 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.221+0000 7fc6eeffd640 1 -- 172.21.15.160:0/2090456708 <== mon.1 v1:172.21.15.169:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (unknown 1482059429 0 984267973) 0x7fc6e8068630 con 0x7fc6f810a5a0 2024-09-16T08:46:14.222 INFO:teuthology.orchestra.run.smithi160.stdout:214748364810 2024-09-16T08:46:14.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.223+0000 7fc700075640 1 -- 172.21.15.160:0/2090456708 >> v1:172.21.15.160:6800/2859204856 conn(0x7fc6e407bc20 legacy=0x7fc6e407e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:14.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.223+0000 7fc700075640 1 -- 172.21.15.160:0/2090456708 >> v1:172.21.15.169:6789/0 conn(0x7fc6f810a5a0 legacy=0x7fc6f81a25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:14.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.223+0000 7fc700075640 1 -- 172.21.15.160:0/2090456708 shutdown_connections 2024-09-16T08:46:14.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.223+0000 7fc700075640 1 -- 172.21.15.160:0/2090456708 >> 172.21.15.160:0/2090456708 conn(0x7fc6f80fd480 msgr2=0x7fc6f8105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:14.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.224+0000 7fc700075640 1 -- 172.21.15.160:0/2090456708 shutdown_connections 2024-09-16T08:46:14.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:14.224+0000 7fc700075640 1 -- 172.21.15.160:0/2090456708 wait complete. 2024-09-16T08:46:14.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:14 smithi160 ceph-mon[30137]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T08:46:14.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2090456708' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-16T08:46:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:14 smithi160 ceph-mon[40028]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T08:46:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:14 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2090456708' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-16T08:46:14.533 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411367 got 77309411369 for osd.2 2024-09-16T08:46:14.534 DEBUG:teuthology.parallel:result is None 2024-09-16T08:46:14.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:14 smithi169 ceph-mon[33465]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T08:46:14.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:14 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2090456708' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-16T08:46:14.963 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:46:15.099 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:46:14 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:46:15.167 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:46:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:46:15.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.625+0000 7f0598442640 1 -- 172.21.15.160:0/2080223620 >> v1:172.21.15.160:6789/0 conn(0x7f059010f240 legacy=0x7f0590111700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:15.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.625+0000 7f0598442640 1 -- 172.21.15.160:0/2080223620 shutdown_connections 2024-09-16T08:46:15.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.625+0000 7f0598442640 1 -- 172.21.15.160:0/2080223620 >> 172.21.15.160:0/2080223620 conn(0x7f05900fe580 msgr2=0x7f05901009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:15.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.625+0000 7f0598442640 1 -- 172.21.15.160:0/2080223620 shutdown_connections 2024-09-16T08:46:15.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.625+0000 7f0598442640 1 -- 172.21.15.160:0/2080223620 wait complete. 2024-09-16T08:46:15.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.626+0000 7f0598442640 1 Processor -- start 2024-09-16T08:46:15.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.626+0000 7f0598442640 1 -- start start 2024-09-16T08:46:15.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.626+0000 7f0598442640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f05901a77f0 con 0x7f059010f240 2024-09-16T08:46:15.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.626+0000 7f0598442640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f05901a89f0 con 0x7f059010b6a0 2024-09-16T08:46:15.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.626+0000 7f0598442640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f05901a9bf0 con 0x7f0590102b30 2024-09-16T08:46:15.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.626+0000 7f05961de640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f0590102b30 0x7f059010e930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:55724/0 (socket says 172.21.15.160:55724) 2024-09-16T08:46:15.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.626+0000 7f05961de640 1 -- 172.21.15.160:0/2778282347 learned_addr learned my addr 172.21.15.160:0/2778282347 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:15.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.626+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1447807150 0 0) 0x7f05901a77f0 con 0x7f059010f240 2024-09-16T08:46:15.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.626+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f057c003160 con 0x7f059010f240 2024-09-16T08:46:15.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.626+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 397196696 0 0) 0x7f05901a9bf0 con 0x7f0590102b30 2024-09-16T08:46:15.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.626+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05901a77f0 con 0x7f0590102b30 2024-09-16T08:46:15.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.626+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3626934583 0 0) 0x7f05901a89f0 con 0x7f059010b6a0 2024-09-16T08:46:15.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.627+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05901a9bf0 con 0x7f059010b6a0 2024-09-16T08:46:15.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.627+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1579731647 0 0) 0x7f057c003160 con 0x7f059010f240 2024-09-16T08:46:15.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.627+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f05901a89f0 con 0x7f059010f240 2024-09-16T08:46:15.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.627+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f0588001930 con 0x7f059010f240 2024-09-16T08:46:15.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.627+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4176673693 0 0) 0x7f05901a77f0 con 0x7f0590102b30 2024-09-16T08:46:15.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.627+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f057c003160 con 0x7f0590102b30 2024-09-16T08:46:15.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.627+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f0584001410 con 0x7f0590102b30 2024-09-16T08:46:15.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.627+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2761708202 0 0) 0x7f05901a89f0 con 0x7f059010f240 2024-09-16T08:46:15.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.627+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 >> v1:172.21.15.160:6790/0 conn(0x7f0590102b30 legacy=0x7f059010e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:15.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.627+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 >> v1:172.21.15.169:6789/0 conn(0x7f059010b6a0 legacy=0x7f05901a2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:15.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.627+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05901aae90 con 0x7f059010f240 2024-09-16T08:46:15.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.627+0000 7f0598442640 1 -- 172.21.15.160:0/2778282347 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f05901a9e20 con 0x7f059010f240 2024-09-16T08:46:15.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.627+0000 7f0598442640 1 -- 172.21.15.160:0/2778282347 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f05901aa450 con 0x7f059010f240 2024-09-16T08:46:15.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.627+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f0588001d80 con 0x7f059010f240 2024-09-16T08:46:15.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.627+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f0588004dd0 con 0x7f059010f240 2024-09-16T08:46:15.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.628+0000 7f0598442640 1 -- 172.21.15.160:0/2778282347 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05740050b0 con 0x7f059010f240 2024-09-16T08:46:15.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.628+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7f0588004220 con 0x7f059010f240 2024-09-16T08:46:15.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.629+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(64..64 src has 1..64) ==== 6172+0+0 (unknown 440296652 0 0) 0x7f0588098060 con 0x7f059010f240 2024-09-16T08:46:15.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.633+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f05880636b0 con 0x7f059010f240 2024-09-16T08:46:15.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.814+0000 7f0598442640 1 -- 172.21.15.160:0/2778282347 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f05740053a0 con 0x7f059010f240 2024-09-16T08:46:15.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.814+0000 7f05837fe640 1 -- 172.21.15.160:0/2778282347 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (unknown 383520633 0 1618599718) 0x7f05880685f0 con 0x7f059010f240 2024-09-16T08:46:15.815 INFO:teuthology.orchestra.run.smithi160.stdout:111669149731 2024-09-16T08:46:15.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.817+0000 7f0598442640 1 -- 172.21.15.160:0/2778282347 >> v1:172.21.15.160:6800/2859204856 conn(0x7f057c07b920 legacy=0x7f057c07dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:15.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.817+0000 7f0598442640 1 -- 172.21.15.160:0/2778282347 >> v1:172.21.15.160:6789/0 conn(0x7f059010f240 legacy=0x7f05901a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:15.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.817+0000 7f0598442640 1 -- 172.21.15.160:0/2778282347 shutdown_connections 2024-09-16T08:46:15.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.817+0000 7f0598442640 1 -- 172.21.15.160:0/2778282347 >> 172.21.15.160:0/2778282347 conn(0x7f05900fe580 msgr2=0x7f0590106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:15.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.817+0000 7f0598442640 1 -- 172.21.15.160:0/2778282347 shutdown_connections 2024-09-16T08:46:15.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:15.817+0000 7f0598442640 1 -- 172.21.15.160:0/2778282347 wait complete. 2024-09-16T08:46:15.982 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953499 got 137438953501 for osd.4 2024-09-16T08:46:15.982 DEBUG:teuthology.parallel:result is None 2024-09-16T08:46:16.567 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:16.567 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:16 smithi160 ceph-mon[30137]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T08:46:16.567 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2778282347' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-16T08:46:16.568 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:16.568 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:16 smithi160 ceph-mon[40028]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T08:46:16.568 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2778282347' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-16T08:46:16.593 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149728 got 111669149731 for osd.3 2024-09-16T08:46:16.594 DEBUG:teuthology.parallel:result is None 2024-09-16T08:46:16.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:16.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:16 smithi169 ceph-mon[33465]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T08:46:16.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2778282347' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-16T08:46:17.216 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364809 got 214748364810 for osd.7 2024-09-16T08:46:17.216 DEBUG:teuthology.parallel:result is None 2024-09-16T08:46:17.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.411+0000 7fb72f7a4640 1 -- 172.21.15.160:0/1805700415 >> v1:172.21.15.160:6790/0 conn(0x7fb72810d8b0 legacy=0x7fb72810fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:17.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.411+0000 7fb72f7a4640 1 -- 172.21.15.160:0/1805700415 shutdown_connections 2024-09-16T08:46:17.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.411+0000 7fb72f7a4640 1 -- 172.21.15.160:0/1805700415 >> 172.21.15.160:0/1805700415 conn(0x7fb728078f70 msgr2=0x7fb728079380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:17.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.411+0000 7fb72f7a4640 1 -- 172.21.15.160:0/1805700415 shutdown_connections 2024-09-16T08:46:17.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.411+0000 7fb72f7a4640 1 -- 172.21.15.160:0/1805700415 wait complete. 2024-09-16T08:46:17.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.411+0000 7fb72f7a4640 1 Processor -- start 2024-09-16T08:46:17.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.411+0000 7fb72f7a4640 1 -- start start 2024-09-16T08:46:17.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb72f7a4640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb7281a5440 con 0x7fb72810d8b0 2024-09-16T08:46:17.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb72f7a4640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb7281a6620 con 0x7fb728101520 2024-09-16T08:46:17.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb72f7a4640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb7281a7800 con 0x7fb728109cb0 2024-09-16T08:46:17.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb72cd3f640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7fb728109cb0 0x7fb72810a7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:43502/0 (socket says 172.21.15.160:43502) 2024-09-16T08:46:17.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb72dd41640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fb72810d8b0 0x7fb7281a3d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:45916/0 (socket says 172.21.15.160:45916) 2024-09-16T08:46:17.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb72cd3f640 1 -- 172.21.15.160:0/2904621025 learned_addr learned my addr 172.21.15.160:0/2904621025 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:17.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1602050352 0 0) 0x7fb7281a7800 con 0x7fb728109cb0 2024-09-16T08:46:17.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7100056d0 con 0x7fb728109cb0 2024-09-16T08:46:17.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3785104939 0 0) 0x7fb7281a5440 con 0x7fb72810d8b0 2024-09-16T08:46:17.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7281a7800 con 0x7fb72810d8b0 2024-09-16T08:46:17.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2338346984 0 0) 0x7fb7100056d0 con 0x7fb728109cb0 2024-09-16T08:46:17.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb7281a5440 con 0x7fb728109cb0 2024-09-16T08:46:17.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb71c0019d0 con 0x7fb728109cb0 2024-09-16T08:46:17.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 67930709 0 0) 0x7fb7281a7800 con 0x7fb72810d8b0 2024-09-16T08:46:17.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb7100056d0 con 0x7fb72810d8b0 2024-09-16T08:46:17.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2775942822 0 0) 0x7fb7281a6620 con 0x7fb728101520 2024-09-16T08:46:17.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7281a7800 con 0x7fb728101520 2024-09-16T08:46:17.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb720001790 con 0x7fb72810d8b0 2024-09-16T08:46:17.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3312668015 0 0) 0x7fb7281a5440 con 0x7fb728109cb0 2024-09-16T08:46:17.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 >> v1:172.21.15.169:6789/0 conn(0x7fb728101520 legacy=0x7fb72810a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:17.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.412+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 >> v1:172.21.15.160:6789/0 conn(0x7fb72810d8b0 legacy=0x7fb7281a3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:17.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.413+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7281a8a80 con 0x7fb728109cb0 2024-09-16T08:46:17.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.413+0000 7fb72f7a4640 1 -- 172.21.15.160:0/2904621025 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb7281a5670 con 0x7fb728109cb0 2024-09-16T08:46:17.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.413+0000 7fb72f7a4640 1 -- 172.21.15.160:0/2904621025 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fb7281a5ca0 con 0x7fb728109cb0 2024-09-16T08:46:17.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.413+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fb71c001c50 con 0x7fb728109cb0 2024-09-16T08:46:17.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.413+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fb71c005ca0 con 0x7fb728109cb0 2024-09-16T08:46:17.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.414+0000 7fb72f7a4640 1 -- 172.21.15.160:0/2904621025 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7140050b0 con 0x7fb728109cb0 2024-09-16T08:46:17.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.414+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7fb71c0038c0 con 0x7fb728109cb0 2024-09-16T08:46:17.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.415+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(64..64 src has 1..64) ==== 6172+0+0 (unknown 440296652 0 0) 0x7fb71c099380 con 0x7fb728109cb0 2024-09-16T08:46:17.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.419+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fb71c0649f0 con 0x7fb728109cb0 2024-09-16T08:46:17.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.612+0000 7fb72f7a4640 1 -- 172.21.15.160:0/2904621025 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb7140053a0 con 0x7fb728109cb0 2024-09-16T08:46:17.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.612+0000 7fb7267fc640 1 -- 172.21.15.160:0/2904621025 <== mon.2 v1:172.21.15.160:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (unknown 574334944 0 190233549) 0x7fb71c069930 con 0x7fb728109cb0 2024-09-16T08:46:17.615 INFO:teuthology.orchestra.run.smithi160.stdout:34359738421 2024-09-16T08:46:17.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.615+0000 7fb72f7a4640 1 -- 172.21.15.160:0/2904621025 >> v1:172.21.15.160:6800/2859204856 conn(0x7fb71007dfe0 legacy=0x7fb7100804a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:17.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.615+0000 7fb72f7a4640 1 -- 172.21.15.160:0/2904621025 >> v1:172.21.15.160:6790/0 conn(0x7fb728109cb0 legacy=0x7fb72810a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:17.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.615+0000 7fb72f7a4640 1 -- 172.21.15.160:0/2904621025 shutdown_connections 2024-09-16T08:46:17.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.615+0000 7fb72f7a4640 1 -- 172.21.15.160:0/2904621025 >> 172.21.15.160:0/2904621025 conn(0x7fb728078f70 msgr2=0x7fb728106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:17.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.615+0000 7fb72f7a4640 1 -- 172.21.15.160:0/2904621025 shutdown_connections 2024-09-16T08:46:17.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:17.615+0000 7fb72f7a4640 1 -- 172.21.15.160:0/2904621025 wait complete. 2024-09-16T08:46:18.262 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738419 got 34359738421 for osd.0 2024-09-16T08:46:18.262 DEBUG:teuthology.parallel:result is None 2024-09-16T08:46:18.263 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T08:46:18.263 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T08:46:18.539 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:46:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:18 smithi169 ceph-mon[33465]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T08:46:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:18 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:18 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2904621025' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-16T08:46:18.661 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:18 smithi160 ceph-mon[30137]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T08:46:18.661 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:18 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:18.661 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:18 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:18.662 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2904621025' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-16T08:46:18.662 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:18 smithi160 ceph-mon[40028]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T08:46:18.662 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:18 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:18.662 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:18 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:18.662 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2904621025' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-16T08:46:19.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.114+0000 7faa1e222640 1 -- 172.21.15.160:0/3692304721 >> v1:172.21.15.160:6790/0 conn(0x7faa18102110 legacy=0x7faa1810fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:19.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.114+0000 7faa1e222640 1 -- 172.21.15.160:0/3692304721 shutdown_connections 2024-09-16T08:46:19.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.114+0000 7faa1e222640 1 -- 172.21.15.160:0/3692304721 >> 172.21.15.160:0/3692304721 conn(0x7faa180fd450 msgr2=0x7faa180ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:19.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.115+0000 7faa1e222640 1 -- 172.21.15.160:0/3692304721 shutdown_connections 2024-09-16T08:46:19.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.115+0000 7faa1e222640 1 -- 172.21.15.160:0/3692304721 wait complete. 2024-09-16T08:46:19.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.115+0000 7faa1e222640 1 Processor -- start 2024-09-16T08:46:19.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.115+0000 7faa1e222640 1 -- start start 2024-09-16T08:46:19.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa1e222640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7faa181a99a0 con 0x7faa18101620 2024-09-16T08:46:19.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa1e222640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7faa181aaba0 con 0x7faa18111450 2024-09-16T08:46:19.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa1e222640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7faa181abda0 con 0x7faa18102110 2024-09-16T08:46:19.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa177fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7faa18101620 0x7faa18110b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:45922/0 (socket says 172.21.15.160:45922) 2024-09-16T08:46:19.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa177fe640 1 -- 172.21.15.160:0/186039312 learned_addr learned my addr 172.21.15.160:0/186039312 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:19.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3349618312 0 0) 0x7faa181a99a0 con 0x7faa18101620 2024-09-16T08:46:19.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa04003160 con 0x7faa18101620 2024-09-16T08:46:19.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2996530796 0 0) 0x7faa181abda0 con 0x7faa18102110 2024-09-16T08:46:19.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa181a99a0 con 0x7faa18102110 2024-09-16T08:46:19.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 150543178 0 0) 0x7faa181aaba0 con 0x7faa18111450 2024-09-16T08:46:19.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa181abda0 con 0x7faa18111450 2024-09-16T08:46:19.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3349983030 0 0) 0x7faa04003160 con 0x7faa18101620 2024-09-16T08:46:19.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7faa181aaba0 con 0x7faa18101620 2024-09-16T08:46:19.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7faa08001a10 con 0x7faa18101620 2024-09-16T08:46:19.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2131535378 0 0) 0x7faa181abda0 con 0x7faa18111450 2024-09-16T08:46:19.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7faa04003160 con 0x7faa18111450 2024-09-16T08:46:19.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7faa100014c0 con 0x7faa18111450 2024-09-16T08:46:19.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4246205572 0 0) 0x7faa181a99a0 con 0x7faa18102110 2024-09-16T08:46:19.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7faa181abda0 con 0x7faa18102110 2024-09-16T08:46:19.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7faa00001630 con 0x7faa18102110 2024-09-16T08:46:19.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 704477540 0 0) 0x7faa181aaba0 con 0x7faa18101620 2024-09-16T08:46:19.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 >> v1:172.21.15.160:6790/0 conn(0x7faa18102110 legacy=0x7faa181a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:19.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 >> v1:172.21.15.169:6789/0 conn(0x7faa18111450 legacy=0x7faa181a80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:19.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.116+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa181ad040 con 0x7faa18101620 2024-09-16T08:46:19.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.117+0000 7faa1e222640 1 -- 172.21.15.160:0/186039312 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7faa181abfd0 con 0x7faa18101620 2024-09-16T08:46:19.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.117+0000 7faa1e222640 1 -- 172.21.15.160:0/186039312 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7faa181ac600 con 0x7faa18101620 2024-09-16T08:46:19.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.117+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7faa08001d30 con 0x7faa18101620 2024-09-16T08:46:19.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.117+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7faa08005ce0 con 0x7faa18101620 2024-09-16T08:46:19.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.118+0000 7faa1e222640 1 -- 172.21.15.160:0/186039312 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f80050b0 con 0x7faa18101620 2024-09-16T08:46:19.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.118+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7faa080038c0 con 0x7faa18101620 2024-09-16T08:46:19.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.119+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(64..64 src has 1..64) ==== 6172+0+0 (unknown 440296652 0 0) 0x7faa08099300 con 0x7faa18101620 2024-09-16T08:46:19.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.123+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7faa08064910 con 0x7faa18101620 2024-09-16T08:46:19.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.314+0000 7faa1e222640 1 -- 172.21.15.160:0/186039312 --> v1:172.21.15.160:6800/2859204856 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9f8000bc0 con 0x7faa0407bc70 2024-09-16T08:46:19.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.320+0000 7faa14ff9640 1 -- 172.21.15.160:0/186039312 <== mgr.34526 v1:172.21.15.160:6800/2859204856 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+346453 (unknown 2965378022 0 1711601086) 0x7fa9f8000bc0 con 0x7faa0407bc70 2024-09-16T08:46:19.325 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:46:19.327 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-09-16T08:46:19.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.325+0000 7faa1e222640 1 -- 172.21.15.160:0/186039312 >> v1:172.21.15.160:6800/2859204856 conn(0x7faa0407bc70 legacy=0x7faa0407e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:19.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.325+0000 7faa1e222640 1 -- 172.21.15.160:0/186039312 >> v1:172.21.15.160:6789/0 conn(0x7faa18101620 legacy=0x7faa18110b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:19.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.325+0000 7faa1e222640 1 -- 172.21.15.160:0/186039312 shutdown_connections 2024-09-16T08:46:19.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.325+0000 7faa1e222640 1 -- 172.21.15.160:0/186039312 >> 172.21.15.160:0/186039312 conn(0x7faa180fd450 msgr2=0x7faa18114900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:19.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.325+0000 7faa1e222640 1 -- 172.21.15.160:0/186039312 shutdown_connections 2024-09-16T08:46:19.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:19.325+0000 7faa1e222640 1 -- 172.21.15.160:0/186039312 wait complete. 2024-09-16T08:46:19.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: Adjusting osd_memory_target on smithi160 to 3760M 2024-09-16T08:46:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:46:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:46:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: Updating smithi160:/etc/ceph/ceph.conf 2024-09-16T08:46:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T08:46:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:46:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: Updating smithi160:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:46:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: Updating smithi169:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:46:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.676 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:19.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:19.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:19.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:19.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: Adjusting osd_memory_target on smithi160 to 3760M 2024-09-16T08:46:19.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:46:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:46:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: Updating smithi160:/etc/ceph/ceph.conf 2024-09-16T08:46:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T08:46:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:46:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: Updating smithi160:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:46:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: Updating smithi169:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:46:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.679 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.679 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.679 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:19.679 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:19.679 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:19.679 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T08:46:19.679 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: Adjusting osd_memory_target on smithi160 to 3760M 2024-09-16T08:46:19.679 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.679 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:46:19.679 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:46:19.679 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: Updating smithi160:/etc/ceph/ceph.conf 2024-09-16T08:46:19.679 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T08:46:19.679 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:46:19.680 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: Updating smithi160:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:46:19.680 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: Updating smithi169:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:46:19.680 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.680 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.680 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.680 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:19.680 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:20.167 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-09-16T08:46:19.039790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":888,"num_read_kb":745,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":239668,"kb_used_data":6812,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":749721548,"statfs":{"total":767960285184,"available":767714865152,"internally_reserved":0,"allocated":6975488,"data_stored":3721177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002650"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.024534+0000","last_change":"2024-09-16T08:45:33.813300+0000","last_active":"2024-09-16T08:46:03.024534+0000","last_peered":"2024-09-16T08:46:03.024534+0000","last_clean":"2024-09-16T08:46:03.024534+0000","last_became_active":"2024-09-16T08:45:33.813044+0000","last_became_peered":"2024-09-16T08:45:33.813044+0000","last_unstale":"2024-09-16T08:46:03.024534+0000","last_undegraded":"2024-09-16T08:46:03.024534+0000","last_fullsized":"2024-09-16T08:46:03.024534+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:14:51.205142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019302+0000","last_change":"2024-09-16T08:45:35.835577+0000","last_active":"2024-09-16T08:46:03.019302+0000","last_peered":"2024-09-16T08:46:03.019302+0000","last_clean":"2024-09-16T08:46:03.019302+0000","last_became_active":"2024-09-16T08:45:35.835280+0000","last_became_peered":"2024-09-16T08:45:35.835280+0000","last_unstale":"2024-09-16T08:46:03.019302+0000","last_undegraded":"2024-09-16T08:46:03.019302+0000","last_fullsized":"2024-09-16T08:46:03.019302+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:21:59.670525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.346018+0000","last_change":"2024-09-16T08:45:37.851433+0000","last_active":"2024-09-16T08:46:03.346018+0000","last_peered":"2024-09-16T08:46:03.346018+0000","last_clean":"2024-09-16T08:46:03.346018+0000","last_became_active":"2024-09-16T08:45:37.846008+0000","last_became_peered":"2024-09-16T08:45:37.846008+0000","last_unstale":"2024-09-16T08:46:03.346018+0000","last_undegraded":"2024-09-16T08:46:03.346018+0000","last_fullsized":"2024-09-16T08:46:03.346018+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:24:50.563483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019361+0000","last_change":"2024-09-16T08:45:39.869890+0000","last_active":"2024-09-16T08:46:03.019361+0000","last_peered":"2024-09-16T08:46:03.019361+0000","last_clean":"2024-09-16T08:46:03.019361+0000","last_became_active":"2024-09-16T08:45:39.869531+0000","last_became_peered":"2024-09-16T08:45:39.869531+0000","last_unstale":"2024-09-16T08:46:03.019361+0000","last_undegraded":"2024-09-16T08:46:03.019361+0000","last_fullsized":"2024-09-16T08:46:03.019361+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:25:31.737119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864425+0000","last_change":"2024-09-16T08:45:39.867250+0000","last_active":"2024-09-16T08:46:03.864425+0000","last_peered":"2024-09-16T08:46:03.864425+0000","last_clean":"2024-09-16T08:46:03.864425+0000","last_became_active":"2024-09-16T08:45:39.866898+0000","last_became_peered":"2024-09-16T08:45:39.866898+0000","last_unstale":"2024-09-16T08:46:03.864425+0000","last_undegraded":"2024-09-16T08:46:03.864425+0000","last_fullsized":"2024-09-16T08:46:03.864425+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:50:50.235631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.865696+0000","last_change":"2024-09-16T08:45:33.812271+0000","last_active":"2024-09-16T08:46:03.865696+0000","last_peered":"2024-09-16T08:46:03.865696+0000","last_clean":"2024-09-16T08:46:03.865696+0000","last_became_active":"2024-09-16T08:45:33.811993+0000","last_became_peered":"2024-09-16T08:45:33.811993+0000","last_unstale":"2024-09-16T08:46:03.865696+0000","last_undegraded":"2024-09-16T08:46:03.865696+0000","last_fullsized":"2024-09-16T08:46:03.865696+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:35.687926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"62'15","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.865999+0000","last_change":"2024-09-16T08:45:35.829196+0000","last_active":"2024-09-16T08:46:03.865999+0000","last_peered":"2024-09-16T08:46:03.865999+0000","last_clean":"2024-09-16T08:46:03.865999+0000","last_became_active":"2024-09-16T08:45:35.827938+0000","last_became_peered":"2024-09-16T08:45:35.827938+0000","last_unstale":"2024-09-16T08:46:03.865999+0000","last_undegraded":"2024-09-16T08:46:03.865999+0000","last_fullsized":"2024-09-16T08:46:03.865999+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:07.260344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019299+0000","last_change":"2024-09-16T08:45:37.846028+0000","last_active":"2024-09-16T08:46:03.019299+0000","last_peered":"2024-09-16T08:46:03.019299+0000","last_clean":"2024-09-16T08:46:03.019299+0000","last_became_active":"2024-09-16T08:45:37.845112+0000","last_became_peered":"2024-09-16T08:45:37.845112+0000","last_unstale":"2024-09-16T08:46:03.019299+0000","last_undegraded":"2024-09-16T08:46:03.019299+0000","last_fullsized":"2024-09-16T08:46:03.019299+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:04.613475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362612+0000","last_change":"2024-09-16T08:45:33.817292+0000","last_active":"2024-09-16T08:46:03.362612+0000","last_peered":"2024-09-16T08:46:03.362612+0000","last_clean":"2024-09-16T08:46:03.362612+0000","last_became_active":"2024-09-16T08:45:33.817001+0000","last_became_peered":"2024-09-16T08:45:33.817001+0000","last_unstale":"2024-09-16T08:46:03.362612+0000","last_undegraded":"2024-09-16T08:46:03.362612+0000","last_fullsized":"2024-09-16T08:46:03.362612+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:17.614320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019593+0000","last_change":"2024-09-16T08:45:35.826899+0000","last_active":"2024-09-16T08:46:03.019593+0000","last_peered":"2024-09-16T08:46:03.019593+0000","last_clean":"2024-09-16T08:46:03.019593+0000","last_became_active":"2024-09-16T08:45:35.826592+0000","last_became_peered":"2024-09-16T08:45:35.826592+0000","last_unstale":"2024-09-16T08:46:03.019593+0000","last_undegraded":"2024-09-16T08:46:03.019593+0000","last_fullsized":"2024-09-16T08:46:03.019593+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:34.217603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362253+0000","last_change":"2024-09-16T08:45:37.848960+0000","last_active":"2024-09-16T08:46:03.362253+0000","last_peered":"2024-09-16T08:46:03.362253+0000","last_clean":"2024-09-16T08:46:03.362253+0000","last_became_active":"2024-09-16T08:45:37.848542+0000","last_became_peered":"2024-09-16T08:45:37.848542+0000","last_unstale":"2024-09-16T08:46:03.362253+0000","last_undegraded":"2024-09-16T08:46:03.362253+0000","last_fullsized":"2024-09-16T08:46:03.362253+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:29.874057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019299+0000","last_change":"2024-09-16T08:45:39.865646+0000","last_active":"2024-09-16T08:46:03.019299+0000","last_peered":"2024-09-16T08:46:03.019299+0000","last_clean":"2024-09-16T08:46:03.019299+0000","last_became_active":"2024-09-16T08:45:39.864665+0000","last_became_peered":"2024-09-16T08:45:39.864665+0000","last_unstale":"2024-09-16T08:46:03.019299+0000","last_undegraded":"2024-09-16T08:46:03.019299+0000","last_fullsized":"2024-09-16T08:46:03.019299+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:25.934182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362345+0000","last_change":"2024-09-16T08:45:33.819294+0000","last_active":"2024-09-16T08:46:03.362345+0000","last_peered":"2024-09-16T08:46:03.362345+0000","last_clean":"2024-09-16T08:46:03.362345+0000","last_became_active":"2024-09-16T08:45:33.818672+0000","last_became_peered":"2024-09-16T08:45:33.818672+0000","last_unstale":"2024-09-16T08:46:03.362345+0000","last_undegraded":"2024-09-16T08:46:03.362345+0000","last_fullsized":"2024-09-16T08:46:03.362345+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:22:56.714569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"62'5","reported_seq":36,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020271+0000","last_change":"2024-09-16T08:45:35.835638+0000","last_active":"2024-09-16T08:46:03.020271+0000","last_peered":"2024-09-16T08:46:03.020271+0000","last_clean":"2024-09-16T08:46:03.020271+0000","last_became_active":"2024-09-16T08:45:35.835328+0000","last_became_peered":"2024-09-16T08:45:35.835328+0000","last_unstale":"2024-09-16T08:46:03.020271+0000","last_undegraded":"2024-09-16T08:46:03.020271+0000","last_fullsized":"2024-09-16T08:46:03.020271+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:22.259226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.367730+0000","last_change":"2024-09-16T08:45:37.846525+0000","last_active":"2024-09-16T08:46:03.367730+0000","last_peered":"2024-09-16T08:46:03.367730+0000","last_clean":"2024-09-16T08:46:03.367730+0000","last_became_active":"2024-09-16T08:45:37.846221+0000","last_became_peered":"2024-09-16T08:45:37.846221+0000","last_unstale":"2024-09-16T08:46:03.367730+0000","last_undegraded":"2024-09-16T08:46:03.367730+0000","last_fullsized":"2024-09-16T08:46:03.367730+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:20:38.234799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362070+0000","last_change":"2024-09-16T08:45:39.872162+0000","last_active":"2024-09-16T08:46:03.362070+0000","last_peered":"2024-09-16T08:46:03.362070+0000","last_clean":"2024-09-16T08:46:03.362070+0000","last_became_active":"2024-09-16T08:45:39.871888+0000","last_became_peered":"2024-09-16T08:45:39.871888+0000","last_unstale":"2024-09-16T08:46:03.362070+0000","last_undegraded":"2024-09-16T08:46:03.362070+0000","last_fullsized":"2024-09-16T08:46:03.362070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:15:43.600755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019856+0000","last_change":"2024-09-16T08:45:39.868502+0000","last_active":"2024-09-16T08:46:03.019856+0000","last_peered":"2024-09-16T08:46:03.019856+0000","last_clean":"2024-09-16T08:46:03.019856+0000","last_became_active":"2024-09-16T08:45:39.868192+0000","last_became_peered":"2024-09-16T08:45:39.868192+0000","last_unstale":"2024-09-16T08:46:03.019856+0000","last_undegraded":"2024-09-16T08:46:03.019856+0000","last_fullsized":"2024-09-16T08:46:03.019856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:10.995950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.024259+0000","last_change":"2024-09-16T08:45:33.811109+0000","last_active":"2024-09-16T08:46:03.024259+0000","last_peered":"2024-09-16T08:46:03.024259+0000","last_clean":"2024-09-16T08:46:03.024259+0000","last_became_active":"2024-09-16T08:45:33.810826+0000","last_became_peered":"2024-09-16T08:45:33.810826+0000","last_unstale":"2024-09-16T08:46:03.024259+0000","last_undegraded":"2024-09-16T08:46:03.024259+0000","last_fullsized":"2024-09-16T08:46:03.024259+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:42.551767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"62'15","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.022687+0000","last_change":"2024-09-16T08:45:35.833831+0000","last_active":"2024-09-16T08:46:03.022687+0000","last_peered":"2024-09-16T08:46:03.022687+0000","last_clean":"2024-09-16T08:46:03.022687+0000","last_became_active":"2024-09-16T08:45:35.833285+0000","last_became_peered":"2024-09-16T08:45:35.833285+0000","last_unstale":"2024-09-16T08:46:03.022687+0000","last_undegraded":"2024-09-16T08:46:03.022687+0000","last_fullsized":"2024-09-16T08:46:03.022687+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:56:10.339005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.346556+0000","last_change":"2024-09-16T08:45:37.853870+0000","last_active":"2024-09-16T08:46:03.346556+0000","last_peered":"2024-09-16T08:46:03.346556+0000","last_clean":"2024-09-16T08:46:03.346556+0000","last_became_active":"2024-09-16T08:45:37.852373+0000","last_became_peered":"2024-09-16T08:45:37.852373+0000","last_unstale":"2024-09-16T08:46:03.346556+0000","last_undegraded":"2024-09-16T08:46:03.346556+0000","last_fullsized":"2024-09-16T08:46:03.346556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:20.086294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.865363+0000","last_change":"2024-09-16T08:45:39.870832+0000","last_active":"2024-09-16T08:46:03.865363+0000","last_peered":"2024-09-16T08:46:03.865363+0000","last_clean":"2024-09-16T08:46:03.865363+0000","last_became_active":"2024-09-16T08:45:39.870528+0000","last_became_peered":"2024-09-16T08:45:39.870528+0000","last_unstale":"2024-09-16T08:46:03.865363+0000","last_undegraded":"2024-09-16T08:46:03.865363+0000","last_fullsized":"2024-09-16T08:46:03.865363+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:55.564948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019797+0000","last_change":"2024-09-16T08:45:33.812741+0000","last_active":"2024-09-16T08:46:03.019797+0000","last_peered":"2024-09-16T08:46:03.019797+0000","last_clean":"2024-09-16T08:46:03.019797+0000","last_became_active":"2024-09-16T08:45:33.812465+0000","last_became_peered":"2024-09-16T08:45:33.812465+0000","last_unstale":"2024-09-16T08:46:03.019797+0000","last_undegraded":"2024-09-16T08:46:03.019797+0000","last_fullsized":"2024-09-16T08:46:03.019797+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:18:41.804184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"62'12","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.865310+0000","last_change":"2024-09-16T08:45:35.835095+0000","last_active":"2024-09-16T08:46:03.865310+0000","last_peered":"2024-09-16T08:46:03.865310+0000","last_clean":"2024-09-16T08:46:03.865310+0000","last_became_active":"2024-09-16T08:45:35.834753+0000","last_became_peered":"2024-09-16T08:45:35.834753+0000","last_unstale":"2024-09-16T08:46:03.865310+0000","last_undegraded":"2024-09-16T08:46:03.865310+0000","last_fullsized":"2024-09-16T08:46:03.865310+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:29.974141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020302+0000","last_change":"2024-09-16T08:45:37.852729+0000","last_active":"2024-09-16T08:46:03.020302+0000","last_peered":"2024-09-16T08:46:03.020302+0000","last_clean":"2024-09-16T08:46:03.020302+0000","last_became_active":"2024-09-16T08:45:37.852416+0000","last_became_peered":"2024-09-16T08:45:37.852416+0000","last_unstale":"2024-09-16T08:46:03.020302+0000","last_undegraded":"2024-09-16T08:46:03.020302+0000","last_fullsized":"2024-09-16T08:46:03.020302+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:39.691495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"62'1","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.367981+0000","last_change":"2024-09-16T08:45:39.866127+0000","last_active":"2024-09-16T08:46:03.367981+0000","last_peered":"2024-09-16T08:46:03.367981+0000","last_clean":"2024-09-16T08:46:03.367981+0000","last_became_active":"2024-09-16T08:45:39.865860+0000","last_became_peered":"2024-09-16T08:45:39.865860+0000","last_unstale":"2024-09-16T08:46:03.367981+0000","last_undegraded":"2024-09-16T08:46:03.367981+0000","last_fullsized":"2024-09-16T08:46:03.367981+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:39.148052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"55'1","reported_seq":31,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.345584+0000","last_change":"2024-09-16T08:45:33.811044+0000","last_active":"2024-09-16T08:46:03.345584+0000","last_peered":"2024-09-16T08:46:03.345584+0000","last_clean":"2024-09-16T08:46:03.345584+0000","last_became_active":"2024-09-16T08:45:33.810752+0000","last_became_peered":"2024-09-16T08:45:33.810752+0000","last_unstale":"2024-09-16T08:46:03.345584+0000","last_undegraded":"2024-09-16T08:46:03.345584+0000","last_fullsized":"2024-09-16T08:46:03.345584+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:00:54.360250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"62'10","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020295+0000","last_change":"2024-09-16T08:45:35.841723+0000","last_active":"2024-09-16T08:46:03.020295+0000","last_peered":"2024-09-16T08:46:03.020295+0000","last_clean":"2024-09-16T08:46:03.020295+0000","last_became_active":"2024-09-16T08:45:35.841366+0000","last_became_peered":"2024-09-16T08:45:35.841366+0000","last_unstale":"2024-09-16T08:46:03.020295+0000","last_undegraded":"2024-09-16T08:46:03.020295+0000","last_fullsized":"2024-09-16T08:46:03.020295+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:00:35.472570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"62'8","reported_seq":43,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.865170+0000","last_change":"2024-09-16T08:45:37.849929+0000","last_active":"2024-09-16T08:46:13.865170+0000","last_peered":"2024-09-16T08:46:13.865170+0000","last_clean":"2024-09-16T08:46:13.865170+0000","last_became_active":"2024-09-16T08:45:37.849092+0000","last_became_peered":"2024-09-16T08:45:37.849092+0000","last_unstale":"2024-09-16T08:46:13.865170+0000","last_undegraded":"2024-09-16T08:46:13.865170+0000","last_fullsized":"2024-09-16T08:46:13.865170+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:17:04.338511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"62'15","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.346627+0000","last_change":"2024-09-16T08:45:35.838344+0000","last_active":"2024-09-16T08:46:03.346627+0000","last_peered":"2024-09-16T08:46:03.346627+0000","last_clean":"2024-09-16T08:46:03.346627+0000","last_became_active":"2024-09-16T08:45:35.838037+0000","last_became_peered":"2024-09-16T08:45:35.838037+0000","last_unstale":"2024-09-16T08:46:03.346627+0000","last_undegraded":"2024-09-16T08:46:03.346627+0000","last_fullsized":"2024-09-16T08:46:03.346627+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:12:53.229643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864415+0000","last_change":"2024-09-16T08:45:33.809967+0000","last_active":"2024-09-16T08:46:03.864415+0000","last_peered":"2024-09-16T08:46:03.864415+0000","last_clean":"2024-09-16T08:46:03.864415+0000","last_became_active":"2024-09-16T08:45:33.809640+0000","last_became_peered":"2024-09-16T08:45:33.809640+0000","last_unstale":"2024-09-16T08:46:03.864415+0000","last_undegraded":"2024-09-16T08:46:03.864415+0000","last_fullsized":"2024-09-16T08:46:03.864415+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:06.774145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"62'8","reported_seq":40,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.865445+0000","last_change":"2024-09-16T08:45:37.845825+0000","last_active":"2024-09-16T08:46:13.865445+0000","last_peered":"2024-09-16T08:46:13.865445+0000","last_clean":"2024-09-16T08:46:13.865445+0000","last_became_active":"2024-09-16T08:45:37.845541+0000","last_became_peered":"2024-09-16T08:45:37.845541+0000","last_unstale":"2024-09-16T08:46:13.865445+0000","last_undegraded":"2024-09-16T08:46:13.865445+0000","last_fullsized":"2024-09-16T08:46:13.865445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:58:25.792250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362358+0000","last_change":"2024-09-16T08:45:39.871072+0000","last_active":"2024-09-16T08:46:03.362358+0000","last_peered":"2024-09-16T08:46:03.362358+0000","last_clean":"2024-09-16T08:46:03.362358+0000","last_became_active":"2024-09-16T08:45:39.870793+0000","last_became_peered":"2024-09-16T08:45:39.870793+0000","last_unstale":"2024-09-16T08:46:03.362358+0000","last_undegraded":"2024-09-16T08:46:03.362358+0000","last_fullsized":"2024-09-16T08:46:03.362358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:18:06.743025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"62'18","reported_seq":58,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864217+0000","last_change":"2024-09-16T08:45:35.840066+0000","last_active":"2024-09-16T08:46:03.864217+0000","last_peered":"2024-09-16T08:46:03.864217+0000","last_clean":"2024-09-16T08:46:03.864217+0000","last_became_active":"2024-09-16T08:45:35.839601+0000","last_became_peered":"2024-09-16T08:45:35.839601+0000","last_unstale":"2024-09-16T08:46:03.864217+0000","last_undegraded":"2024-09-16T08:46:03.864217+0000","last_fullsized":"2024-09-16T08:46:03.864217+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:04.727428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.865950+0000","last_change":"2024-09-16T08:45:33.811448+0000","last_active":"2024-09-16T08:46:03.865950+0000","last_peered":"2024-09-16T08:46:03.865950+0000","last_clean":"2024-09-16T08:46:03.865950+0000","last_became_active":"2024-09-16T08:45:33.811187+0000","last_became_peered":"2024-09-16T08:45:33.811187+0000","last_unstale":"2024-09-16T08:46:03.865950+0000","last_undegraded":"2024-09-16T08:46:03.865950+0000","last_fullsized":"2024-09-16T08:46:03.865950+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:41:41.933623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019349+0000","last_change":"2024-09-16T08:45:37.851147+0000","last_active":"2024-09-16T08:46:03.019349+0000","last_peered":"2024-09-16T08:46:03.019349+0000","last_clean":"2024-09-16T08:46:03.019349+0000","last_became_active":"2024-09-16T08:45:37.850717+0000","last_became_peered":"2024-09-16T08:45:37.850717+0000","last_unstale":"2024-09-16T08:46:03.019349+0000","last_undegraded":"2024-09-16T08:46:03.019349+0000","last_fullsized":"2024-09-16T08:46:03.019349+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:31:35.555632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019566+0000","last_change":"2024-09-16T08:45:39.867089+0000","last_active":"2024-09-16T08:46:03.019566+0000","last_peered":"2024-09-16T08:46:03.019566+0000","last_clean":"2024-09-16T08:46:03.019566+0000","last_became_active":"2024-09-16T08:45:39.866784+0000","last_became_peered":"2024-09-16T08:45:39.866784+0000","last_unstale":"2024-09-16T08:46:03.019566+0000","last_undegraded":"2024-09-16T08:46:03.019566+0000","last_fullsized":"2024-09-16T08:46:03.019566+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:47.876056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"62'14","reported_seq":47,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019358+0000","last_change":"2024-09-16T08:45:35.826507+0000","last_active":"2024-09-16T08:46:03.019358+0000","last_peered":"2024-09-16T08:46:03.019358+0000","last_clean":"2024-09-16T08:46:03.019358+0000","last_became_active":"2024-09-16T08:45:35.826160+0000","last_became_peered":"2024-09-16T08:45:35.826160+0000","last_unstale":"2024-09-16T08:46:03.019358+0000","last_undegraded":"2024-09-16T08:46:03.019358+0000","last_fullsized":"2024-09-16T08:46:03.019358+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:55:24.026157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"55'1","reported_seq":31,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.021062+0000","last_change":"2024-09-16T08:45:33.808737+0000","last_active":"2024-09-16T08:46:03.021062+0000","last_peered":"2024-09-16T08:46:03.021062+0000","last_clean":"2024-09-16T08:46:03.021062+0000","last_became_active":"2024-09-16T08:45:33.808424+0000","last_became_peered":"2024-09-16T08:45:33.808424+0000","last_unstale":"2024-09-16T08:46:03.021062+0000","last_undegraded":"2024-09-16T08:46:03.021062+0000","last_fullsized":"2024-09-16T08:46:03.021062+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:41.655775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"62'8","reported_seq":40,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.864959+0000","last_change":"2024-09-16T08:45:37.855366+0000","last_active":"2024-09-16T08:46:13.864959+0000","last_peered":"2024-09-16T08:46:13.864959+0000","last_clean":"2024-09-16T08:46:13.864959+0000","last_became_active":"2024-09-16T08:45:37.855008+0000","last_became_peered":"2024-09-16T08:45:37.855008+0000","last_unstale":"2024-09-16T08:46:13.864959+0000","last_undegraded":"2024-09-16T08:46:13.864959+0000","last_fullsized":"2024-09-16T08:46:13.864959+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:32.170198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.367835+0000","last_change":"2024-09-16T08:45:39.866879+0000","last_active":"2024-09-16T08:46:03.367835+0000","last_peered":"2024-09-16T08:46:03.367835+0000","last_clean":"2024-09-16T08:46:03.367835+0000","last_became_active":"2024-09-16T08:45:39.866594+0000","last_became_peered":"2024-09-16T08:45:39.866594+0000","last_unstale":"2024-09-16T08:46:03.367835+0000","last_undegraded":"2024-09-16T08:46:03.367835+0000","last_fullsized":"2024-09-16T08:46:03.367835+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:16:39.764595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"62'10","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.345776+0000","last_change":"2024-09-16T08:45:35.835159+0000","last_active":"2024-09-16T08:46:03.345776+0000","last_peered":"2024-09-16T08:46:03.345776+0000","last_clean":"2024-09-16T08:46:03.345776+0000","last_became_active":"2024-09-16T08:45:35.833871+0000","last_became_peered":"2024-09-16T08:45:35.833871+0000","last_unstale":"2024-09-16T08:46:03.345776+0000","last_undegraded":"2024-09-16T08:46:03.345776+0000","last_fullsized":"2024-09-16T08:46:03.345776+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:37:29.528117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362691+0000","last_change":"2024-09-16T08:45:33.817698+0000","last_active":"2024-09-16T08:46:03.362691+0000","last_peered":"2024-09-16T08:46:03.362691+0000","last_clean":"2024-09-16T08:46:03.362691+0000","last_became_active":"2024-09-16T08:45:33.817381+0000","last_became_peered":"2024-09-16T08:45:33.817381+0000","last_unstale":"2024-09-16T08:46:03.362691+0000","last_undegraded":"2024-09-16T08:46:03.362691+0000","last_fullsized":"2024-09-16T08:46:03.362691+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:11:54.619759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"62'8","reported_seq":40,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.864970+0000","last_change":"2024-09-16T08:45:37.849922+0000","last_active":"2024-09-16T08:46:13.864970+0000","last_peered":"2024-09-16T08:46:13.864970+0000","last_clean":"2024-09-16T08:46:13.864970+0000","last_became_active":"2024-09-16T08:45:37.849621+0000","last_became_peered":"2024-09-16T08:45:37.849621+0000","last_unstale":"2024-09-16T08:46:13.864970+0000","last_undegraded":"2024-09-16T08:46:13.864970+0000","last_fullsized":"2024-09-16T08:46:13.864970+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:21:07.076075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019444+0000","last_change":"2024-09-16T08:45:39.864584+0000","last_active":"2024-09-16T08:46:03.019444+0000","last_peered":"2024-09-16T08:46:03.019444+0000","last_clean":"2024-09-16T08:46:03.019444+0000","last_became_active":"2024-09-16T08:45:39.863463+0000","last_became_peered":"2024-09-16T08:45:39.863463+0000","last_unstale":"2024-09-16T08:46:03.019444+0000","last_undegraded":"2024-09-16T08:46:03.019444+0000","last_fullsized":"2024-09-16T08:46:03.019444+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:27.795309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"62'19","reported_seq":62,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.023975+0000","last_change":"2024-09-16T08:45:35.838887+0000","last_active":"2024-09-16T08:46:03.023975+0000","last_peered":"2024-09-16T08:46:03.023975+0000","last_clean":"2024-09-16T08:46:03.023975+0000","last_became_active":"2024-09-16T08:45:35.838565+0000","last_became_peered":"2024-09-16T08:45:35.838565+0000","last_unstale":"2024-09-16T08:46:03.023975+0000","last_undegraded":"2024-09-16T08:46:03.023975+0000","last_fullsized":"2024-09-16T08:46:03.023975+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:51.681132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.346247+0000","last_change":"2024-09-16T08:45:33.812693+0000","last_active":"2024-09-16T08:46:03.346247+0000","last_peered":"2024-09-16T08:46:03.346247+0000","last_clean":"2024-09-16T08:46:03.346247+0000","last_became_active":"2024-09-16T08:45:33.811644+0000","last_became_peered":"2024-09-16T08:45:33.811644+0000","last_unstale":"2024-09-16T08:46:03.346247+0000","last_undegraded":"2024-09-16T08:46:03.346247+0000","last_fullsized":"2024-09-16T08:46:03.346247+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:45:46.664593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.412555+0000","last_change":"2024-09-16T08:45:37.848576+0000","last_active":"2024-09-16T08:46:03.412555+0000","last_peered":"2024-09-16T08:46:03.412555+0000","last_clean":"2024-09-16T08:46:03.412555+0000","last_became_active":"2024-09-16T08:45:37.848298+0000","last_became_peered":"2024-09-16T08:45:37.848298+0000","last_unstale":"2024-09-16T08:46:03.412555+0000","last_undegraded":"2024-09-16T08:46:03.412555+0000","last_fullsized":"2024-09-16T08:46:03.412555+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:09:18.833845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.345407+0000","last_change":"2024-09-16T08:45:39.866848+0000","last_active":"2024-09-16T08:46:03.345407+0000","last_peered":"2024-09-16T08:46:03.345407+0000","last_clean":"2024-09-16T08:46:03.345407+0000","last_became_active":"2024-09-16T08:45:39.866559+0000","last_became_peered":"2024-09-16T08:45:39.866559+0000","last_unstale":"2024-09-16T08:46:03.345407+0000","last_undegraded":"2024-09-16T08:46:03.345407+0000","last_fullsized":"2024-09-16T08:46:03.345407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:19:44.414025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"62'28","reported_seq":84,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.864959+0000","last_change":"2024-09-16T08:45:35.832443+0000","last_active":"2024-09-16T08:46:13.864959+0000","last_peered":"2024-09-16T08:46:13.864959+0000","last_clean":"2024-09-16T08:46:13.864959+0000","last_became_active":"2024-09-16T08:45:35.832110+0000","last_became_peered":"2024-09-16T08:45:35.832110+0000","last_unstale":"2024-09-16T08:46:13.864959+0000","last_undegraded":"2024-09-16T08:46:13.864959+0000","last_fullsized":"2024-09-16T08:46:13.864959+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:16:55.691286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020650+0000","last_change":"2024-09-16T08:45:33.815029+0000","last_active":"2024-09-16T08:46:03.020650+0000","last_peered":"2024-09-16T08:46:03.020650+0000","last_clean":"2024-09-16T08:46:03.020650+0000","last_became_active":"2024-09-16T08:45:33.814734+0000","last_became_peered":"2024-09-16T08:45:33.814734+0000","last_unstale":"2024-09-16T08:46:03.020650+0000","last_undegraded":"2024-09-16T08:46:03.020650+0000","last_fullsized":"2024-09-16T08:46:03.020650+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:30.492050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"57'2","reported_seq":40,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362310+0000","last_change":"2024-09-16T08:45:35.824559+0000","last_active":"2024-09-16T08:46:03.362310+0000","last_peered":"2024-09-16T08:46:03.362310+0000","last_clean":"2024-09-16T08:46:03.362310+0000","last_became_active":"2024-09-16T08:45:32.804280+0000","last_became_peered":"2024-09-16T08:45:32.804280+0000","last_unstale":"2024-09-16T08:46:03.362310+0000","last_undegraded":"2024-09-16T08:46:03.362310+0000","last_fullsized":"2024-09-16T08:46:03.362310+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:31.787970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:31.787970+0000","last_clean_scrub_stamp":"2024-09-16T08:45:31.787970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:09:07.773521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001552082,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.023281+0000","last_change":"2024-09-16T08:45:37.845569+0000","last_active":"2024-09-16T08:46:03.023281+0000","last_peered":"2024-09-16T08:46:03.023281+0000","last_clean":"2024-09-16T08:46:03.023281+0000","last_became_active":"2024-09-16T08:45:37.845272+0000","last_became_peered":"2024-09-16T08:45:37.845272+0000","last_unstale":"2024-09-16T08:46:03.023281+0000","last_undegraded":"2024-09-16T08:46:03.023281+0000","last_fullsized":"2024-09-16T08:46:03.023281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:51.576180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"62'1","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.865338+0000","last_change":"2024-09-16T08:45:39.872662+0000","last_active":"2024-09-16T08:46:03.865338+0000","last_peered":"2024-09-16T08:46:03.865338+0000","last_clean":"2024-09-16T08:46:03.865338+0000","last_became_active":"2024-09-16T08:45:39.872337+0000","last_became_peered":"2024-09-16T08:45:39.872337+0000","last_unstale":"2024-09-16T08:46:03.865338+0000","last_undegraded":"2024-09-16T08:46:03.865338+0000","last_fullsized":"2024-09-16T08:46:03.865338+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:59.658604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"62'13","reported_seq":53,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.346751+0000","last_change":"2024-09-16T08:45:35.834310+0000","last_active":"2024-09-16T08:46:03.346751+0000","last_peered":"2024-09-16T08:46:03.346751+0000","last_clean":"2024-09-16T08:46:03.346751+0000","last_became_active":"2024-09-16T08:45:35.833790+0000","last_became_peered":"2024-09-16T08:45:35.833790+0000","last_unstale":"2024-09-16T08:46:03.346751+0000","last_undegraded":"2024-09-16T08:46:03.346751+0000","last_fullsized":"2024-09-16T08:46:03.346751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:38:14.928129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.345506+0000","last_change":"2024-09-16T08:45:33.813084+0000","last_active":"2024-09-16T08:46:03.345506+0000","last_peered":"2024-09-16T08:46:03.345506+0000","last_clean":"2024-09-16T08:46:03.345506+0000","last_became_active":"2024-09-16T08:45:33.812820+0000","last_became_peered":"2024-09-16T08:45:33.812820+0000","last_unstale":"2024-09-16T08:46:03.345506+0000","last_undegraded":"2024-09-16T08:46:03.345506+0000","last_fullsized":"2024-09-16T08:46:03.345506+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:54.637276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"55'1","reported_seq":39,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.021750+0000","last_change":"2024-09-16T08:45:35.818992+0000","last_active":"2024-09-16T08:46:03.021750+0000","last_peered":"2024-09-16T08:46:03.021750+0000","last_clean":"2024-09-16T08:46:03.021750+0000","last_became_active":"2024-09-16T08:45:32.802160+0000","last_became_peered":"2024-09-16T08:45:32.802160+0000","last_unstale":"2024-09-16T08:46:03.021750+0000","last_undegraded":"2024-09-16T08:46:03.021750+0000","last_fullsized":"2024-09-16T08:46:03.021750+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:31.787970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:31.787970+0000","last_clean_scrub_stamp":"2024-09-16T08:45:31.787970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:40.625663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060482500000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.022319+0000","last_change":"2024-09-16T08:45:37.846890+0000","last_active":"2024-09-16T08:46:03.022319+0000","last_peered":"2024-09-16T08:46:03.022319+0000","last_clean":"2024-09-16T08:46:03.022319+0000","last_became_active":"2024-09-16T08:45:37.846582+0000","last_became_peered":"2024-09-16T08:45:37.846582+0000","last_unstale":"2024-09-16T08:46:03.022319+0000","last_undegraded":"2024-09-16T08:46:03.022319+0000","last_fullsized":"2024-09-16T08:46:03.022319+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:22.783666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.368113+0000","last_change":"2024-09-16T08:45:39.865107+0000","last_active":"2024-09-16T08:46:03.368113+0000","last_peered":"2024-09-16T08:46:03.368113+0000","last_clean":"2024-09-16T08:46:03.368113+0000","last_became_active":"2024-09-16T08:45:39.864836+0000","last_became_peered":"2024-09-16T08:45:39.864836+0000","last_unstale":"2024-09-16T08:46:03.368113+0000","last_undegraded":"2024-09-16T08:46:03.368113+0000","last_fullsized":"2024-09-16T08:46:03.368113+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:38.365579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"62'12","reported_seq":44,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020797+0000","last_change":"2024-09-16T08:45:35.834103+0000","last_active":"2024-09-16T08:46:03.020797+0000","last_peered":"2024-09-16T08:46:03.020797+0000","last_clean":"2024-09-16T08:46:03.020797+0000","last_became_active":"2024-09-16T08:45:35.833797+0000","last_became_peered":"2024-09-16T08:45:35.833797+0000","last_unstale":"2024-09-16T08:46:03.020797+0000","last_undegraded":"2024-09-16T08:46:03.020797+0000","last_fullsized":"2024-09-16T08:46:03.020797+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:00:11.203579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019424+0000","last_change":"2024-09-16T08:45:33.810293+0000","last_active":"2024-09-16T08:46:03.019424+0000","last_peered":"2024-09-16T08:46:03.019424+0000","last_clean":"2024-09-16T08:46:03.019424+0000","last_became_active":"2024-09-16T08:45:33.809995+0000","last_became_peered":"2024-09-16T08:45:33.809995+0000","last_unstale":"2024-09-16T08:46:03.019424+0000","last_undegraded":"2024-09-16T08:46:03.019424+0000","last_fullsized":"2024-09-16T08:46:03.019424+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:46:05.431730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":81,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:17.731660+0000","last_change":"2024-09-16T08:45:35.837001+0000","last_active":"2024-09-16T08:46:17.731660+0000","last_peered":"2024-09-16T08:46:17.731660+0000","last_clean":"2024-09-16T08:46:17.731660+0000","last_became_active":"2024-09-16T08:45:32.805284+0000","last_became_peered":"2024-09-16T08:45:32.805284+0000","last_unstale":"2024-09-16T08:46:17.731660+0000","last_undegraded":"2024-09-16T08:46:17.731660+0000","last_fullsized":"2024-09-16T08:46:17.731660+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:31.787970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:31.787970+0000","last_clean_scrub_stamp":"2024-09-16T08:45:31.787970+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:06:19.733606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093979999999999997,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":40,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362749+0000","last_change":"2024-09-16T08:45:37.853088+0000","last_active":"2024-09-16T08:46:03.362749+0000","last_peered":"2024-09-16T08:46:03.362749+0000","last_clean":"2024-09-16T08:46:03.362749+0000","last_became_active":"2024-09-16T08:45:37.852639+0000","last_became_peered":"2024-09-16T08:45:37.852639+0000","last_unstale":"2024-09-16T08:46:03.362749+0000","last_undegraded":"2024-09-16T08:46:03.362749+0000","last_fullsized":"2024-09-16T08:46:03.362749+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:54.021042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.345615+0000","last_change":"2024-09-16T08:45:39.869824+0000","last_active":"2024-09-16T08:46:03.345615+0000","last_peered":"2024-09-16T08:46:03.345615+0000","last_clean":"2024-09-16T08:46:03.345615+0000","last_became_active":"2024-09-16T08:45:39.869497+0000","last_became_peered":"2024-09-16T08:45:39.869497+0000","last_unstale":"2024-09-16T08:46:03.345615+0000","last_undegraded":"2024-09-16T08:46:03.345615+0000","last_fullsized":"2024-09-16T08:46:03.345615+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:49:47.403086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"62'16","reported_seq":58,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.865222+0000","last_change":"2024-09-16T08:45:35.839016+0000","last_active":"2024-09-16T08:46:13.865222+0000","last_peered":"2024-09-16T08:46:13.865222+0000","last_clean":"2024-09-16T08:46:13.865222+0000","last_became_active":"2024-09-16T08:45:35.838667+0000","last_became_peered":"2024-09-16T08:45:35.838667+0000","last_unstale":"2024-09-16T08:46:13.865222+0000","last_undegraded":"2024-09-16T08:46:13.865222+0000","last_fullsized":"2024-09-16T08:46:13.865222+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:52:20.141633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864591+0000","last_change":"2024-09-16T08:45:33.813160+0000","last_active":"2024-09-16T08:46:03.864591+0000","last_peered":"2024-09-16T08:46:03.864591+0000","last_clean":"2024-09-16T08:46:03.864591+0000","last_became_active":"2024-09-16T08:45:33.812868+0000","last_became_peered":"2024-09-16T08:45:33.812868+0000","last_unstale":"2024-09-16T08:46:03.864591+0000","last_undegraded":"2024-09-16T08:46:03.864591+0000","last_fullsized":"2024-09-16T08:46:03.864591+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:02.449848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"64'79","reported_seq":67,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:05.341213+0000","last_change":"2024-09-16T08:45:25.077647+0000","last_active":"2024-09-16T08:46:05.341213+0000","last_peered":"2024-09-16T08:46:05.341213+0000","last_clean":"2024-09-16T08:46:05.341213+0000","last_became_active":"2024-09-16T08:45:25.071716+0000","last_became_peered":"2024-09-16T08:45:25.071716+0000","last_unstale":"2024-09-16T08:46:05.341213+0000","last_undegraded":"2024-09-16T08:46:05.341213+0000","last_fullsized":"2024-09-16T08:46:05.341213+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:43:02.998358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:43:02.998358+0000","last_clean_scrub_stamp":"2024-09-16T08:43:02.998358+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:50.593602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.368058+0000","last_change":"2024-09-16T08:45:37.847599+0000","last_active":"2024-09-16T08:46:03.368058+0000","last_peered":"2024-09-16T08:46:03.368058+0000","last_clean":"2024-09-16T08:46:03.368058+0000","last_became_active":"2024-09-16T08:45:37.846998+0000","last_became_peered":"2024-09-16T08:45:37.846998+0000","last_unstale":"2024-09-16T08:46:03.368058+0000","last_undegraded":"2024-09-16T08:46:03.368058+0000","last_fullsized":"2024-09-16T08:46:03.368058+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:38.641395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362668+0000","last_change":"2024-09-16T08:45:39.872581+0000","last_active":"2024-09-16T08:46:03.362668+0000","last_peered":"2024-09-16T08:46:03.362668+0000","last_clean":"2024-09-16T08:46:03.362668+0000","last_became_active":"2024-09-16T08:45:39.871964+0000","last_became_peered":"2024-09-16T08:45:39.871964+0000","last_unstale":"2024-09-16T08:46:03.362668+0000","last_undegraded":"2024-09-16T08:46:03.362668+0000","last_fullsized":"2024-09-16T08:46:03.362668+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:21:10.767288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"62'11","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019944+0000","last_change":"2024-09-16T08:45:35.834626+0000","last_active":"2024-09-16T08:46:03.019944+0000","last_peered":"2024-09-16T08:46:03.019944+0000","last_clean":"2024-09-16T08:46:03.019944+0000","last_became_active":"2024-09-16T08:45:35.833959+0000","last_became_peered":"2024-09-16T08:45:35.833959+0000","last_unstale":"2024-09-16T08:46:03.019944+0000","last_undegraded":"2024-09-16T08:46:03.019944+0000","last_fullsized":"2024-09-16T08:46:03.019944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:10.384496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019601+0000","last_change":"2024-09-16T08:45:33.813379+0000","last_active":"2024-09-16T08:46:03.019601+0000","last_peered":"2024-09-16T08:46:03.019601+0000","last_clean":"2024-09-16T08:46:03.019601+0000","last_became_active":"2024-09-16T08:45:33.813115+0000","last_became_peered":"2024-09-16T08:45:33.813115+0000","last_unstale":"2024-09-16T08:46:03.019601+0000","last_undegraded":"2024-09-16T08:46:03.019601+0000","last_fullsized":"2024-09-16T08:46:03.019601+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:47:55.377999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362627+0000","last_change":"2024-09-16T08:45:37.851256+0000","last_active":"2024-09-16T08:46:03.362627+0000","last_peered":"2024-09-16T08:46:03.362627+0000","last_clean":"2024-09-16T08:46:03.362627+0000","last_became_active":"2024-09-16T08:45:37.850975+0000","last_became_peered":"2024-09-16T08:45:37.850975+0000","last_unstale":"2024-09-16T08:46:03.362627+0000","last_undegraded":"2024-09-16T08:46:03.362627+0000","last_fullsized":"2024-09-16T08:46:03.362627+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:28.209730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864414+0000","last_change":"2024-09-16T08:45:39.868671+0000","last_active":"2024-09-16T08:46:03.864414+0000","last_peered":"2024-09-16T08:46:03.864414+0000","last_clean":"2024-09-16T08:46:03.864414+0000","last_became_active":"2024-09-16T08:45:39.868367+0000","last_became_peered":"2024-09-16T08:45:39.868367+0000","last_unstale":"2024-09-16T08:46:03.864414+0000","last_undegraded":"2024-09-16T08:46:03.864414+0000","last_fullsized":"2024-09-16T08:46:03.864414+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:00:15.182398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"62'17","reported_seq":54,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020100+0000","last_change":"2024-09-16T08:45:35.833654+0000","last_active":"2024-09-16T08:46:03.020100+0000","last_peered":"2024-09-16T08:46:03.020100+0000","last_clean":"2024-09-16T08:46:03.020100+0000","last_became_active":"2024-09-16T08:45:35.832032+0000","last_became_peered":"2024-09-16T08:45:35.832032+0000","last_unstale":"2024-09-16T08:46:03.020100+0000","last_undegraded":"2024-09-16T08:46:03.020100+0000","last_fullsized":"2024-09-16T08:46:03.020100+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:40.179999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.412841+0000","last_change":"2024-09-16T08:45:33.814328+0000","last_active":"2024-09-16T08:46:03.412841+0000","last_peered":"2024-09-16T08:46:03.412841+0000","last_clean":"2024-09-16T08:46:03.412841+0000","last_became_active":"2024-09-16T08:45:33.814010+0000","last_became_peered":"2024-09-16T08:45:33.814010+0000","last_unstale":"2024-09-16T08:46:03.412841+0000","last_undegraded":"2024-09-16T08:46:03.412841+0000","last_fullsized":"2024-09-16T08:46:03.412841+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:53:02.457216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.345708+0000","last_change":"2024-09-16T08:45:37.854068+0000","last_active":"2024-09-16T08:46:03.345708+0000","last_peered":"2024-09-16T08:46:03.345708+0000","last_clean":"2024-09-16T08:46:03.345708+0000","last_became_active":"2024-09-16T08:45:37.853565+0000","last_became_peered":"2024-09-16T08:45:37.853565+0000","last_unstale":"2024-09-16T08:46:03.345708+0000","last_undegraded":"2024-09-16T08:46:03.345708+0000","last_fullsized":"2024-09-16T08:46:03.345708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:47.205588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.022507+0000","last_change":"2024-09-16T08:45:39.871548+0000","last_active":"2024-09-16T08:46:03.022507+0000","last_peered":"2024-09-16T08:46:03.022507+0000","last_clean":"2024-09-16T08:46:03.022507+0000","last_became_active":"2024-09-16T08:45:39.871231+0000","last_became_peered":"2024-09-16T08:45:39.871231+0000","last_unstale":"2024-09-16T08:46:03.022507+0000","last_undegraded":"2024-09-16T08:46:03.022507+0000","last_fullsized":"2024-09-16T08:46:03.022507+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:53.365040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"62'10","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020349+0000","last_change":"2024-09-16T08:45:35.828922+0000","last_active":"2024-09-16T08:46:03.020349+0000","last_peered":"2024-09-16T08:46:03.020349+0000","last_clean":"2024-09-16T08:46:03.020349+0000","last_became_active":"2024-09-16T08:45:35.828478+0000","last_became_peered":"2024-09-16T08:45:35.828478+0000","last_unstale":"2024-09-16T08:46:03.020349+0000","last_undegraded":"2024-09-16T08:46:03.020349+0000","last_fullsized":"2024-09-16T08:46:03.020349+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:51.679150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"55'1","reported_seq":36,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864829+0000","last_change":"2024-09-16T08:45:33.810637+0000","last_active":"2024-09-16T08:46:03.864829+0000","last_peered":"2024-09-16T08:46:03.864829+0000","last_clean":"2024-09-16T08:46:03.864829+0000","last_became_active":"2024-09-16T08:45:33.810348+0000","last_became_peered":"2024-09-16T08:45:33.810348+0000","last_unstale":"2024-09-16T08:46:03.864829+0000","last_undegraded":"2024-09-16T08:46:03.864829+0000","last_fullsized":"2024-09-16T08:46:03.864829+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:43:54.969984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"62'8","reported_seq":40,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.864970+0000","last_change":"2024-09-16T08:45:37.847357+0000","last_active":"2024-09-16T08:46:13.864970+0000","last_peered":"2024-09-16T08:46:13.864970+0000","last_clean":"2024-09-16T08:46:13.864970+0000","last_became_active":"2024-09-16T08:45:37.847049+0000","last_became_peered":"2024-09-16T08:45:37.847049+0000","last_unstale":"2024-09-16T08:46:13.864970+0000","last_undegraded":"2024-09-16T08:46:13.864970+0000","last_fullsized":"2024-09-16T08:46:13.864970+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:12:49.100003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019733+0000","last_change":"2024-09-16T08:45:39.869759+0000","last_active":"2024-09-16T08:46:03.019733+0000","last_peered":"2024-09-16T08:46:03.019733+0000","last_clean":"2024-09-16T08:46:03.019733+0000","last_became_active":"2024-09-16T08:45:39.869494+0000","last_became_peered":"2024-09-16T08:45:39.869494+0000","last_unstale":"2024-09-16T08:46:03.019733+0000","last_undegraded":"2024-09-16T08:46:03.019733+0000","last_fullsized":"2024-09-16T08:46:03.019733+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:18.189832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.023865+0000","last_change":"2024-09-16T08:45:35.833441+0000","last_active":"2024-09-16T08:46:03.023865+0000","last_peered":"2024-09-16T08:46:03.023865+0000","last_clean":"2024-09-16T08:46:03.023865+0000","last_became_active":"2024-09-16T08:45:35.833127+0000","last_became_peered":"2024-09-16T08:45:35.833127+0000","last_unstale":"2024-09-16T08:46:03.023865+0000","last_undegraded":"2024-09-16T08:46:03.023865+0000","last_fullsized":"2024-09-16T08:46:03.023865+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:45:35.228973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362406+0000","last_change":"2024-09-16T08:45:33.818979+0000","last_active":"2024-09-16T08:46:03.362406+0000","last_peered":"2024-09-16T08:46:03.362406+0000","last_clean":"2024-09-16T08:46:03.362406+0000","last_became_active":"2024-09-16T08:45:33.818114+0000","last_became_peered":"2024-09-16T08:45:33.818114+0000","last_unstale":"2024-09-16T08:46:03.362406+0000","last_undegraded":"2024-09-16T08:46:03.362406+0000","last_fullsized":"2024-09-16T08:46:03.362406+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:25.701180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.022092+0000","last_change":"2024-09-16T08:45:37.850266+0000","last_active":"2024-09-16T08:46:03.022092+0000","last_peered":"2024-09-16T08:46:03.022092+0000","last_clean":"2024-09-16T08:46:03.022092+0000","last_became_active":"2024-09-16T08:45:37.849966+0000","last_became_peered":"2024-09-16T08:45:37.849966+0000","last_unstale":"2024-09-16T08:46:03.022092+0000","last_undegraded":"2024-09-16T08:46:03.022092+0000","last_fullsized":"2024-09-16T08:46:03.022092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:23:21.782733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019266+0000","last_change":"2024-09-16T08:45:39.864769+0000","last_active":"2024-09-16T08:46:03.019266+0000","last_peered":"2024-09-16T08:46:03.019266+0000","last_clean":"2024-09-16T08:46:03.019266+0000","last_became_active":"2024-09-16T08:45:39.863928+0000","last_became_peered":"2024-09-16T08:45:39.863928+0000","last_unstale":"2024-09-16T08:46:03.019266+0000","last_undegraded":"2024-09-16T08:46:03.019266+0000","last_fullsized":"2024-09-16T08:46:03.019266+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:35.905425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"62'19","reported_seq":57,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.412658+0000","last_change":"2024-09-16T08:45:35.842013+0000","last_active":"2024-09-16T08:46:03.412658+0000","last_peered":"2024-09-16T08:46:03.412658+0000","last_clean":"2024-09-16T08:46:03.412658+0000","last_became_active":"2024-09-16T08:45:35.840965+0000","last_became_peered":"2024-09-16T08:45:35.840965+0000","last_unstale":"2024-09-16T08:46:03.412658+0000","last_undegraded":"2024-09-16T08:46:03.412658+0000","last_fullsized":"2024-09-16T08:46:03.412658+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:13.342411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.368427+0000","last_change":"2024-09-16T08:45:33.814098+0000","last_active":"2024-09-16T08:46:03.368427+0000","last_peered":"2024-09-16T08:46:03.368427+0000","last_clean":"2024-09-16T08:46:03.368427+0000","last_became_active":"2024-09-16T08:45:33.813825+0000","last_became_peered":"2024-09-16T08:45:33.813825+0000","last_unstale":"2024-09-16T08:46:03.368427+0000","last_undegraded":"2024-09-16T08:46:03.368427+0000","last_fullsized":"2024-09-16T08:46:03.368427+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:56:39.972190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.021932+0000","last_change":"2024-09-16T08:45:37.848540+0000","last_active":"2024-09-16T08:46:03.021932+0000","last_peered":"2024-09-16T08:46:03.021932+0000","last_clean":"2024-09-16T08:46:03.021932+0000","last_became_active":"2024-09-16T08:45:37.848203+0000","last_became_peered":"2024-09-16T08:45:37.848203+0000","last_unstale":"2024-09-16T08:46:03.021932+0000","last_undegraded":"2024-09-16T08:46:03.021932+0000","last_fullsized":"2024-09-16T08:46:03.021932+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:34:14.906084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.367758+0000","last_change":"2024-09-16T08:45:39.864881+0000","last_active":"2024-09-16T08:46:03.367758+0000","last_peered":"2024-09-16T08:46:03.367758+0000","last_clean":"2024-09-16T08:46:03.367758+0000","last_became_active":"2024-09-16T08:45:39.864574+0000","last_became_peered":"2024-09-16T08:45:39.864574+0000","last_unstale":"2024-09-16T08:46:03.367758+0000","last_undegraded":"2024-09-16T08:46:03.367758+0000","last_fullsized":"2024-09-16T08:46:03.367758+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:40:28.412183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"62'12","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019539+0000","last_change":"2024-09-16T08:45:35.832462+0000","last_active":"2024-09-16T08:46:03.019539+0000","last_peered":"2024-09-16T08:46:03.019539+0000","last_clean":"2024-09-16T08:46:03.019539+0000","last_became_active":"2024-09-16T08:45:35.832023+0000","last_became_peered":"2024-09-16T08:45:35.832023+0000","last_unstale":"2024-09-16T08:46:03.019539+0000","last_undegraded":"2024-09-16T08:46:03.019539+0000","last_fullsized":"2024-09-16T08:46:03.019539+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:56:11.648976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.368588+0000","last_change":"2024-09-16T08:45:33.815821+0000","last_active":"2024-09-16T08:46:03.368588+0000","last_peered":"2024-09-16T08:46:03.368588+0000","last_clean":"2024-09-16T08:46:03.368588+0000","last_became_active":"2024-09-16T08:45:33.814856+0000","last_became_peered":"2024-09-16T08:45:33.814856+0000","last_unstale":"2024-09-16T08:46:03.368588+0000","last_undegraded":"2024-09-16T08:46:03.368588+0000","last_fullsized":"2024-09-16T08:46:03.368588+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:05.651367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.021458+0000","last_change":"2024-09-16T08:45:37.851977+0000","last_active":"2024-09-16T08:46:03.021458+0000","last_peered":"2024-09-16T08:46:03.021458+0000","last_clean":"2024-09-16T08:46:03.021458+0000","last_became_active":"2024-09-16T08:45:37.851671+0000","last_became_peered":"2024-09-16T08:45:37.851671+0000","last_unstale":"2024-09-16T08:46:03.021458+0000","last_undegraded":"2024-09-16T08:46:03.021458+0000","last_fullsized":"2024-09-16T08:46:03.021458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:24:10.007023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864468+0000","last_change":"2024-09-16T08:45:39.865121+0000","last_active":"2024-09-16T08:46:03.864468+0000","last_peered":"2024-09-16T08:46:03.864468+0000","last_clean":"2024-09-16T08:46:03.864468+0000","last_became_active":"2024-09-16T08:45:39.864805+0000","last_became_peered":"2024-09-16T08:45:39.864805+0000","last_unstale":"2024-09-16T08:46:03.864468+0000","last_undegraded":"2024-09-16T08:46:03.864468+0000","last_fullsized":"2024-09-16T08:46:03.864468+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:55:57.377752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"62'15","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.361977+0000","last_change":"2024-09-16T08:45:35.837361+0000","last_active":"2024-09-16T08:46:03.361977+0000","last_peered":"2024-09-16T08:46:03.361977+0000","last_clean":"2024-09-16T08:46:03.361977+0000","last_became_active":"2024-09-16T08:45:35.837007+0000","last_became_peered":"2024-09-16T08:45:35.837007+0000","last_unstale":"2024-09-16T08:46:03.361977+0000","last_undegraded":"2024-09-16T08:46:03.361977+0000","last_fullsized":"2024-09-16T08:46:03.361977+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:22.135031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"55'2","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.368593+0000","last_change":"2024-09-16T08:45:33.815616+0000","last_active":"2024-09-16T08:46:03.368593+0000","last_peered":"2024-09-16T08:46:03.368593+0000","last_clean":"2024-09-16T08:46:03.368593+0000","last_became_active":"2024-09-16T08:45:33.815224+0000","last_became_peered":"2024-09-16T08:45:33.815224+0000","last_unstale":"2024-09-16T08:46:03.368593+0000","last_undegraded":"2024-09-16T08:46:03.368593+0000","last_fullsized":"2024-09-16T08:46:03.368593+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:51:46.230172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"62'8","reported_seq":40,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.865132+0000","last_change":"2024-09-16T08:45:37.846741+0000","last_active":"2024-09-16T08:46:13.865132+0000","last_peered":"2024-09-16T08:46:13.865132+0000","last_clean":"2024-09-16T08:46:13.865132+0000","last_became_active":"2024-09-16T08:45:37.846299+0000","last_became_peered":"2024-09-16T08:45:37.846299+0000","last_unstale":"2024-09-16T08:46:13.865132+0000","last_undegraded":"2024-09-16T08:46:13.865132+0000","last_fullsized":"2024-09-16T08:46:13.865132+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:59:57.377332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362052+0000","last_change":"2024-09-16T08:45:39.868506+0000","last_active":"2024-09-16T08:46:03.362052+0000","last_peered":"2024-09-16T08:46:03.362052+0000","last_clean":"2024-09-16T08:46:03.362052+0000","last_became_active":"2024-09-16T08:45:39.868238+0000","last_became_peered":"2024-09-16T08:45:39.868238+0000","last_unstale":"2024-09-16T08:46:03.362052+0000","last_undegraded":"2024-09-16T08:46:03.362052+0000","last_fullsized":"2024-09-16T08:46:03.362052+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:46:25.394963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"55'1","reported_seq":36,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.412619+0000","last_change":"2024-09-16T08:45:33.816048+0000","last_active":"2024-09-16T08:46:03.412619+0000","last_peered":"2024-09-16T08:46:03.412619+0000","last_clean":"2024-09-16T08:46:03.412619+0000","last_became_active":"2024-09-16T08:45:33.815745+0000","last_became_peered":"2024-09-16T08:45:33.815745+0000","last_unstale":"2024-09-16T08:46:03.412619+0000","last_undegraded":"2024-09-16T08:46:03.412619+0000","last_fullsized":"2024-09-16T08:46:03.412619+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:09.838323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"62'6","reported_seq":35,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.866010+0000","last_change":"2024-09-16T08:45:35.838918+0000","last_active":"2024-09-16T08:46:03.866010+0000","last_peered":"2024-09-16T08:46:03.866010+0000","last_clean":"2024-09-16T08:46:03.866010+0000","last_became_active":"2024-09-16T08:45:35.838531+0000","last_became_peered":"2024-09-16T08:45:35.838531+0000","last_unstale":"2024-09-16T08:46:03.866010+0000","last_undegraded":"2024-09-16T08:46:03.866010+0000","last_fullsized":"2024-09-16T08:46:03.866010+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:51:02.630687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362568+0000","last_change":"2024-09-16T08:45:37.851852+0000","last_active":"2024-09-16T08:46:03.362568+0000","last_peered":"2024-09-16T08:46:03.362568+0000","last_clean":"2024-09-16T08:46:03.362568+0000","last_became_active":"2024-09-16T08:45:37.851371+0000","last_became_peered":"2024-09-16T08:45:37.851371+0000","last_unstale":"2024-09-16T08:46:03.362568+0000","last_undegraded":"2024-09-16T08:46:03.362568+0000","last_fullsized":"2024-09-16T08:46:03.362568+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:13:03.345415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.367999+0000","last_change":"2024-09-16T08:45:39.871805+0000","last_active":"2024-09-16T08:46:03.367999+0000","last_peered":"2024-09-16T08:46:03.367999+0000","last_clean":"2024-09-16T08:46:03.367999+0000","last_became_active":"2024-09-16T08:45:39.871531+0000","last_became_peered":"2024-09-16T08:45:39.871531+0000","last_unstale":"2024-09-16T08:46:03.367999+0000","last_undegraded":"2024-09-16T08:46:03.367999+0000","last_fullsized":"2024-09-16T08:46:03.367999+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:36:51.283999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020930+0000","last_change":"2024-09-16T08:45:35.840819+0000","last_active":"2024-09-16T08:46:03.020930+0000","last_peered":"2024-09-16T08:46:03.020930+0000","last_clean":"2024-09-16T08:46:03.020930+0000","last_became_active":"2024-09-16T08:45:35.840490+0000","last_became_peered":"2024-09-16T08:45:35.840490+0000","last_unstale":"2024-09-16T08:46:03.020930+0000","last_undegraded":"2024-09-16T08:46:03.020930+0000","last_fullsized":"2024-09-16T08:46:03.020930+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:44.776012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.367952+0000","last_change":"2024-09-16T08:45:33.814341+0000","last_active":"2024-09-16T08:46:03.367952+0000","last_peered":"2024-09-16T08:46:03.367952+0000","last_clean":"2024-09-16T08:46:03.367952+0000","last_became_active":"2024-09-16T08:45:33.813903+0000","last_became_peered":"2024-09-16T08:45:33.813903+0000","last_unstale":"2024-09-16T08:46:03.367952+0000","last_undegraded":"2024-09-16T08:46:03.367952+0000","last_fullsized":"2024-09-16T08:46:03.367952+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:10:41.555349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.865448+0000","last_change":"2024-09-16T08:45:37.856872+0000","last_active":"2024-09-16T08:46:03.865448+0000","last_peered":"2024-09-16T08:46:03.865448+0000","last_clean":"2024-09-16T08:46:03.865448+0000","last_became_active":"2024-09-16T08:45:37.856561+0000","last_became_peered":"2024-09-16T08:45:37.856561+0000","last_unstale":"2024-09-16T08:46:03.865448+0000","last_undegraded":"2024-09-16T08:46:03.865448+0000","last_fullsized":"2024-09-16T08:46:03.865448+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:35.748066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.023730+0000","last_change":"2024-09-16T08:45:39.870005+0000","last_active":"2024-09-16T08:46:03.023730+0000","last_peered":"2024-09-16T08:46:03.023730+0000","last_clean":"2024-09-16T08:46:03.023730+0000","last_became_active":"2024-09-16T08:45:39.869613+0000","last_became_peered":"2024-09-16T08:45:39.869613+0000","last_unstale":"2024-09-16T08:46:03.023730+0000","last_undegraded":"2024-09-16T08:46:03.023730+0000","last_fullsized":"2024-09-16T08:46:03.023730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:43.815457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362565+0000","last_change":"2024-09-16T08:45:35.841050+0000","last_active":"2024-09-16T08:46:03.362565+0000","last_peered":"2024-09-16T08:46:03.362565+0000","last_clean":"2024-09-16T08:46:03.362565+0000","last_became_active":"2024-09-16T08:45:35.840690+0000","last_became_peered":"2024-09-16T08:45:35.840690+0000","last_unstale":"2024-09-16T08:46:03.362565+0000","last_undegraded":"2024-09-16T08:46:03.362565+0000","last_fullsized":"2024-09-16T08:46:03.362565+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:41.045869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.023399+0000","last_change":"2024-09-16T08:45:33.811462+0000","last_active":"2024-09-16T08:46:03.023399+0000","last_peered":"2024-09-16T08:46:03.023399+0000","last_clean":"2024-09-16T08:46:03.023399+0000","last_became_active":"2024-09-16T08:45:33.811171+0000","last_became_peered":"2024-09-16T08:45:33.811171+0000","last_unstale":"2024-09-16T08:46:03.023399+0000","last_undegraded":"2024-09-16T08:46:03.023399+0000","last_fullsized":"2024-09-16T08:46:03.023399+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:36.378011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"62'8","reported_seq":43,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.865186+0000","last_change":"2024-09-16T08:45:37.851783+0000","last_active":"2024-09-16T08:46:13.865186+0000","last_peered":"2024-09-16T08:46:13.865186+0000","last_clean":"2024-09-16T08:46:13.865186+0000","last_became_active":"2024-09-16T08:45:37.851444+0000","last_became_peered":"2024-09-16T08:45:37.851444+0000","last_unstale":"2024-09-16T08:46:13.865186+0000","last_undegraded":"2024-09-16T08:46:13.865186+0000","last_fullsized":"2024-09-16T08:46:13.865186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:45:15.301516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020597+0000","last_change":"2024-09-16T08:45:39.870668+0000","last_active":"2024-09-16T08:46:03.020597+0000","last_peered":"2024-09-16T08:46:03.020597+0000","last_clean":"2024-09-16T08:46:03.020597+0000","last_became_active":"2024-09-16T08:45:39.870415+0000","last_became_peered":"2024-09-16T08:45:39.870415+0000","last_unstale":"2024-09-16T08:46:03.020597+0000","last_undegraded":"2024-09-16T08:46:03.020597+0000","last_fullsized":"2024-09-16T08:46:03.020597+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:55.494362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"62'10","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864813+0000","last_change":"2024-09-16T08:45:35.839930+0000","last_active":"2024-09-16T08:46:03.864813+0000","last_peered":"2024-09-16T08:46:03.864813+0000","last_clean":"2024-09-16T08:46:03.864813+0000","last_became_active":"2024-09-16T08:45:35.839635+0000","last_became_peered":"2024-09-16T08:45:35.839635+0000","last_unstale":"2024-09-16T08:46:03.864813+0000","last_undegraded":"2024-09-16T08:46:03.864813+0000","last_fullsized":"2024-09-16T08:46:03.864813+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:27:34.133426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.368217+0000","last_change":"2024-09-16T08:45:33.814268+0000","last_active":"2024-09-16T08:46:03.368217+0000","last_peered":"2024-09-16T08:46:03.368217+0000","last_clean":"2024-09-16T08:46:03.368217+0000","last_became_active":"2024-09-16T08:45:33.813539+0000","last_became_peered":"2024-09-16T08:45:33.813539+0000","last_unstale":"2024-09-16T08:46:03.368217+0000","last_undegraded":"2024-09-16T08:46:03.368217+0000","last_fullsized":"2024-09-16T08:46:03.368217+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:56.773850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"62'8","reported_seq":40,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.865247+0000","last_change":"2024-09-16T08:45:37.849710+0000","last_active":"2024-09-16T08:46:13.865247+0000","last_peered":"2024-09-16T08:46:13.865247+0000","last_clean":"2024-09-16T08:46:13.865247+0000","last_became_active":"2024-09-16T08:45:37.849378+0000","last_became_peered":"2024-09-16T08:45:37.849378+0000","last_unstale":"2024-09-16T08:46:13.865247+0000","last_undegraded":"2024-09-16T08:46:13.865247+0000","last_fullsized":"2024-09-16T08:46:13.865247+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:22:57.496083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020022+0000","last_change":"2024-09-16T08:45:39.863788+0000","last_active":"2024-09-16T08:46:03.020022+0000","last_peered":"2024-09-16T08:46:03.020022+0000","last_clean":"2024-09-16T08:46:03.020022+0000","last_became_active":"2024-09-16T08:45:39.863428+0000","last_became_peered":"2024-09-16T08:45:39.863428+0000","last_unstale":"2024-09-16T08:46:03.020022+0000","last_undegraded":"2024-09-16T08:46:03.020022+0000","last_fullsized":"2024-09-16T08:46:03.020022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:29.146892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"62'11","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020377+0000","last_change":"2024-09-16T08:45:35.833430+0000","last_active":"2024-09-16T08:46:03.020377+0000","last_peered":"2024-09-16T08:46:03.020377+0000","last_clean":"2024-09-16T08:46:03.020377+0000","last_became_active":"2024-09-16T08:45:35.833112+0000","last_became_peered":"2024-09-16T08:45:35.833112+0000","last_unstale":"2024-09-16T08:46:03.020377+0000","last_undegraded":"2024-09-16T08:46:03.020377+0000","last_fullsized":"2024-09-16T08:46:03.020377+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:36:41.826634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020471+0000","last_change":"2024-09-16T08:45:33.817133+0000","last_active":"2024-09-16T08:46:03.020471+0000","last_peered":"2024-09-16T08:46:03.020471+0000","last_clean":"2024-09-16T08:46:03.020471+0000","last_became_active":"2024-09-16T08:45:33.816860+0000","last_became_peered":"2024-09-16T08:45:33.816860+0000","last_unstale":"2024-09-16T08:46:03.020471+0000","last_undegraded":"2024-09-16T08:46:03.020471+0000","last_fullsized":"2024-09-16T08:46:03.020471+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:26.123965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.345632+0000","last_change":"2024-09-16T08:45:37.855458+0000","last_active":"2024-09-16T08:46:03.345632+0000","last_peered":"2024-09-16T08:46:03.345632+0000","last_clean":"2024-09-16T08:46:03.345632+0000","last_became_active":"2024-09-16T08:45:37.854292+0000","last_became_peered":"2024-09-16T08:45:37.854292+0000","last_unstale":"2024-09-16T08:46:03.345632+0000","last_undegraded":"2024-09-16T08:46:03.345632+0000","last_fullsized":"2024-09-16T08:46:03.345632+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:01:10.548237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864676+0000","last_change":"2024-09-16T08:45:39.869637+0000","last_active":"2024-09-16T08:46:03.864676+0000","last_peered":"2024-09-16T08:46:03.864676+0000","last_clean":"2024-09-16T08:46:03.864676+0000","last_became_active":"2024-09-16T08:45:39.869352+0000","last_became_peered":"2024-09-16T08:45:39.869352+0000","last_unstale":"2024-09-16T08:46:03.864676+0000","last_undegraded":"2024-09-16T08:46:03.864676+0000","last_fullsized":"2024-09-16T08:46:03.864676+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:44:46.751050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.345893+0000","last_change":"2024-09-16T08:45:35.833229+0000","last_active":"2024-09-16T08:46:03.345893+0000","last_peered":"2024-09-16T08:46:03.345893+0000","last_clean":"2024-09-16T08:46:03.345893+0000","last_became_active":"2024-09-16T08:45:35.832945+0000","last_became_peered":"2024-09-16T08:45:35.832945+0000","last_unstale":"2024-09-16T08:46:03.345893+0000","last_undegraded":"2024-09-16T08:46:03.345893+0000","last_fullsized":"2024-09-16T08:46:03.345893+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:08.228692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.368350+0000","last_change":"2024-09-16T08:45:33.815884+0000","last_active":"2024-09-16T08:46:03.368350+0000","last_peered":"2024-09-16T08:46:03.368350+0000","last_clean":"2024-09-16T08:46:03.368350+0000","last_became_active":"2024-09-16T08:45:33.815590+0000","last_became_peered":"2024-09-16T08:45:33.815590+0000","last_unstale":"2024-09-16T08:46:03.368350+0000","last_undegraded":"2024-09-16T08:46:03.368350+0000","last_fullsized":"2024-09-16T08:46:03.368350+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:04.048331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864577+0000","last_change":"2024-09-16T08:45:37.854982+0000","last_active":"2024-09-16T08:46:03.864577+0000","last_peered":"2024-09-16T08:46:03.864577+0000","last_clean":"2024-09-16T08:46:03.864577+0000","last_became_active":"2024-09-16T08:45:37.854701+0000","last_became_peered":"2024-09-16T08:45:37.854701+0000","last_unstale":"2024-09-16T08:46:03.864577+0000","last_undegraded":"2024-09-16T08:46:03.864577+0000","last_fullsized":"2024-09-16T08:46:03.864577+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:05.952109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019184+0000","last_change":"2024-09-16T08:45:39.866984+0000","last_active":"2024-09-16T08:46:03.019184+0000","last_peered":"2024-09-16T08:46:03.019184+0000","last_clean":"2024-09-16T08:46:03.019184+0000","last_became_active":"2024-09-16T08:45:39.866635+0000","last_became_peered":"2024-09-16T08:45:39.866635+0000","last_unstale":"2024-09-16T08:46:03.019184+0000","last_undegraded":"2024-09-16T08:46:03.019184+0000","last_fullsized":"2024-09-16T08:46:03.019184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:01:03.825861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"62'11","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864364+0000","last_change":"2024-09-16T08:45:35.837782+0000","last_active":"2024-09-16T08:46:03.864364+0000","last_peered":"2024-09-16T08:46:03.864364+0000","last_clean":"2024-09-16T08:46:03.864364+0000","last_became_active":"2024-09-16T08:45:35.837399+0000","last_became_peered":"2024-09-16T08:45:35.837399+0000","last_unstale":"2024-09-16T08:46:03.864364+0000","last_undegraded":"2024-09-16T08:46:03.864364+0000","last_fullsized":"2024-09-16T08:46:03.864364+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:48:40.456091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362648+0000","last_change":"2024-09-16T08:45:33.818022+0000","last_active":"2024-09-16T08:46:03.362648+0000","last_peered":"2024-09-16T08:46:03.362648+0000","last_clean":"2024-09-16T08:46:03.362648+0000","last_became_active":"2024-09-16T08:45:33.817721+0000","last_became_peered":"2024-09-16T08:45:33.817721+0000","last_unstale":"2024-09-16T08:46:03.362648+0000","last_undegraded":"2024-09-16T08:46:03.362648+0000","last_fullsized":"2024-09-16T08:46:03.362648+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:51:58.540936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.367569+0000","last_change":"2024-09-16T08:45:37.846757+0000","last_active":"2024-09-16T08:46:03.367569+0000","last_peered":"2024-09-16T08:46:03.367569+0000","last_clean":"2024-09-16T08:46:03.367569+0000","last_became_active":"2024-09-16T08:45:37.846293+0000","last_became_peered":"2024-09-16T08:45:37.846293+0000","last_unstale":"2024-09-16T08:46:03.367569+0000","last_undegraded":"2024-09-16T08:46:03.367569+0000","last_fullsized":"2024-09-16T08:46:03.367569+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:56.864171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864728+0000","last_change":"2024-09-16T08:45:39.868192+0000","last_active":"2024-09-16T08:46:03.864728+0000","last_peered":"2024-09-16T08:46:03.864728+0000","last_clean":"2024-09-16T08:46:03.864728+0000","last_became_active":"2024-09-16T08:45:39.867893+0000","last_became_peered":"2024-09-16T08:45:39.867893+0000","last_unstale":"2024-09-16T08:46:03.864728+0000","last_undegraded":"2024-09-16T08:46:03.864728+0000","last_fullsized":"2024-09-16T08:46:03.864728+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:07.059601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"62'4","reported_seq":32,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864185+0000","last_change":"2024-09-16T08:45:35.835301+0000","last_active":"2024-09-16T08:46:03.864185+0000","last_peered":"2024-09-16T08:46:03.864185+0000","last_clean":"2024-09-16T08:46:03.864185+0000","last_became_active":"2024-09-16T08:45:35.834963+0000","last_became_peered":"2024-09-16T08:45:35.834963+0000","last_unstale":"2024-09-16T08:46:03.864185+0000","last_undegraded":"2024-09-16T08:46:03.864185+0000","last_fullsized":"2024-09-16T08:46:03.864185+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:29.824770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.024570+0000","last_change":"2024-09-16T08:45:33.809757+0000","last_active":"2024-09-16T08:46:03.024570+0000","last_peered":"2024-09-16T08:46:03.024570+0000","last_clean":"2024-09-16T08:46:03.024570+0000","last_became_active":"2024-09-16T08:45:33.809444+0000","last_became_peered":"2024-09-16T08:45:33.809444+0000","last_unstale":"2024-09-16T08:46:03.024570+0000","last_undegraded":"2024-09-16T08:46:03.024570+0000","last_fullsized":"2024-09-16T08:46:03.024570+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:10.444170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.412547+0000","last_change":"2024-09-16T08:45:37.849382+0000","last_active":"2024-09-16T08:46:03.412547+0000","last_peered":"2024-09-16T08:46:03.412547+0000","last_clean":"2024-09-16T08:46:03.412547+0000","last_became_active":"2024-09-16T08:45:37.849095+0000","last_became_peered":"2024-09-16T08:45:37.849095+0000","last_unstale":"2024-09-16T08:46:03.412547+0000","last_undegraded":"2024-09-16T08:46:03.412547+0000","last_fullsized":"2024-09-16T08:46:03.412547+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:14:49.842211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.367698+0000","last_change":"2024-09-16T08:45:39.872542+0000","last_active":"2024-09-16T08:46:03.367698+0000","last_peered":"2024-09-16T08:46:03.367698+0000","last_clean":"2024-09-16T08:46:03.367698+0000","last_became_active":"2024-09-16T08:45:39.872265+0000","last_became_peered":"2024-09-16T08:45:39.872265+0000","last_unstale":"2024-09-16T08:46:03.367698+0000","last_undegraded":"2024-09-16T08:46:03.367698+0000","last_fullsized":"2024-09-16T08:46:03.367698+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:38.307316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.346269+0000","last_change":"2024-09-16T08:45:39.871358+0000","last_active":"2024-09-16T08:46:03.346269+0000","last_peered":"2024-09-16T08:46:03.346269+0000","last_clean":"2024-09-16T08:46:03.346269+0000","last_became_active":"2024-09-16T08:45:39.871018+0000","last_became_peered":"2024-09-16T08:45:39.871018+0000","last_unstale":"2024-09-16T08:46:03.346269+0000","last_undegraded":"2024-09-16T08:46:03.346269+0000","last_fullsized":"2024-09-16T08:46:03.346269+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:09:14.465197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864373+0000","last_change":"2024-09-16T08:45:33.805599+0000","last_active":"2024-09-16T08:46:03.864373+0000","last_peered":"2024-09-16T08:46:03.864373+0000","last_clean":"2024-09-16T08:46:03.864373+0000","last_became_active":"2024-09-16T08:45:33.805262+0000","last_became_peered":"2024-09-16T08:45:33.805262+0000","last_unstale":"2024-09-16T08:46:03.864373+0000","last_undegraded":"2024-09-16T08:46:03.864373+0000","last_fullsized":"2024-09-16T08:46:03.864373+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:15:24.051369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"62'11","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.413104+0000","last_change":"2024-09-16T08:45:35.833205+0000","last_active":"2024-09-16T08:46:03.413104+0000","last_peered":"2024-09-16T08:46:03.413104+0000","last_clean":"2024-09-16T08:46:03.413104+0000","last_became_active":"2024-09-16T08:45:35.832892+0000","last_became_peered":"2024-09-16T08:45:35.832892+0000","last_unstale":"2024-09-16T08:46:03.413104+0000","last_undegraded":"2024-09-16T08:46:03.413104+0000","last_fullsized":"2024-09-16T08:46:03.413104+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:07:29.314959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020132+0000","last_change":"2024-09-16T08:45:37.846900+0000","last_active":"2024-09-16T08:46:03.020132+0000","last_peered":"2024-09-16T08:46:03.020132+0000","last_clean":"2024-09-16T08:46:03.020132+0000","last_became_active":"2024-09-16T08:45:37.846616+0000","last_became_peered":"2024-09-16T08:45:37.846616+0000","last_unstale":"2024-09-16T08:46:03.020132+0000","last_undegraded":"2024-09-16T08:46:03.020132+0000","last_fullsized":"2024-09-16T08:46:03.020132+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:47.867331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":35,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":50,"seq":214748364812,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32396,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712756,"statfs":{"total":95995035648,"available":95961862144,"internally_reserved":0,"allocated":1241088,"data_stored":833976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561041,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1220608,"data_stored":833314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757271,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717508,"statfs":{"total":95995035648,"available":95966728192,"internally_reserved":0,"allocated":630784,"data_stored":243305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953503,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31832,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713320,"statfs":{"total":95995035648,"available":95962439680,"internally_reserved":0,"allocated":659456,"data_stored":244121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149732,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27680,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717472,"statfs":{"total":95995035648,"available":95966691328,"internally_reserved":0,"allocated":667648,"data_stored":244407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411371,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27632,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717520,"statfs":{"total":95995035648,"available":95966740480,"internally_reserved":0,"allocated":618496,"data_stored":242658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31860,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713292,"statfs":{"total":95995035648,"available":95962411008,"internally_reserved":0,"allocated":688128,"data_stored":244295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32404,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712748,"statfs":{"total":95995035648,"available":95961853952,"internally_reserved":0,"allocated":1249280,"data_stored":835101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T08:46:20.174 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T08:46:20.426 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:46:20.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:20 smithi160 ceph-mon[30137]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:46:20.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:20 smithi160 ceph-mon[30137]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:46:20.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:20 smithi160 ceph-mon[30137]: Updating smithi160:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:46:20.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:20 smithi160 ceph-mon[30137]: Updating smithi169:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:46:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:20 smithi160 ceph-mon[30137]: Deploying daemon alertmanager.a on smithi160 2024-09-16T08:46:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:20 smithi160 ceph-mon[30137]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:46:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:20 smithi160 ceph-mon[30137]: from='client.14787 v1:172.21.15.160:0/186039312' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:46:20.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:20 smithi160 ceph-mon[40028]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:46:20.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:20 smithi160 ceph-mon[40028]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:46:20.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:20 smithi160 ceph-mon[40028]: Updating smithi160:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:46:20.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:20 smithi160 ceph-mon[40028]: Updating smithi169:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:46:20.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:20 smithi160 ceph-mon[40028]: Deploying daemon alertmanager.a on smithi160 2024-09-16T08:46:20.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:20 smithi160 ceph-mon[40028]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:46:20.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:20 smithi160 ceph-mon[40028]: from='client.14787 v1:172.21.15.160:0/186039312' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:46:20.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:20 smithi169 ceph-mon[33465]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:46:20.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:20 smithi169 ceph-mon[33465]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:46:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:20 smithi169 ceph-mon[33465]: Updating smithi160:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:46:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:20 smithi169 ceph-mon[33465]: Updating smithi169:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:46:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:20 smithi169 ceph-mon[33465]: Deploying daemon alertmanager.a on smithi160 2024-09-16T08:46:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:20 smithi169 ceph-mon[33465]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:46:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:20 smithi169 ceph-mon[33465]: from='client.14787 v1:172.21.15.160:0/186039312' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:46:21.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.252+0000 7f1aad9b6640 1 -- 172.21.15.160:0/3135578478 >> v1:172.21.15.160:6790/0 conn(0x7f1aa810a590 legacy=0x7f1aa810c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.252+0000 7f1aad9b6640 1 -- 172.21.15.160:0/3135578478 shutdown_connections 2024-09-16T08:46:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.252+0000 7f1aad9b6640 1 -- 172.21.15.160:0/3135578478 >> 172.21.15.160:0/3135578478 conn(0x7f1aa80fd4b0 msgr2=0x7f1aa80ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.252+0000 7f1aad9b6640 1 -- 172.21.15.160:0/3135578478 shutdown_connections 2024-09-16T08:46:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.252+0000 7f1aad9b6640 1 -- 172.21.15.160:0/3135578478 wait complete. 2024-09-16T08:46:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.253+0000 7f1aad9b6640 1 Processor -- start 2024-09-16T08:46:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.253+0000 7f1aad9b6640 1 -- start start 2024-09-16T08:46:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aad9b6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1aa81a9930 con 0x7f1aa8101a20 2024-09-16T08:46:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aad9b6640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1aa81aab30 con 0x7f1aa810e130 2024-09-16T08:46:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aad9b6640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1aa81abd30 con 0x7f1aa810a590 2024-09-16T08:46:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aa67fc640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f1aa810a590 0x7f1aa81a4980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:43556/0 (socket says 172.21.15.160:43556) 2024-09-16T08:46:21.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aa6ffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f1aa8101a20 0x7f1aa810d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:45942/0 (socket says 172.21.15.160:45942) 2024-09-16T08:46:21.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aa67fc640 1 -- 172.21.15.160:0/149761308 learned_addr learned my addr 172.21.15.160:0/149761308 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:21.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1996012707 0 0) 0x7f1aa81a9930 con 0x7f1aa8101a20 2024-09-16T08:46:21.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a90003520 con 0x7f1aa8101a20 2024-09-16T08:46:21.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1596707465 0 0) 0x7f1aa81abd30 con 0x7f1aa810a590 2024-09-16T08:46:21.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1aa81a9930 con 0x7f1aa810a590 2024-09-16T08:46:21.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1653694299 0 0) 0x7f1aa81aab30 con 0x7f1aa810e130 2024-09-16T08:46:21.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1aa81abd30 con 0x7f1aa810e130 2024-09-16T08:46:21.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1343917304 0 0) 0x7f1a90003520 con 0x7f1aa8101a20 2024-09-16T08:46:21.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1aa81aab30 con 0x7f1aa8101a20 2024-09-16T08:46:21.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f1a98001310 con 0x7f1aa8101a20 2024-09-16T08:46:21.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 140760444 0 0) 0x7f1aa81a9930 con 0x7f1aa810a590 2024-09-16T08:46:21.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1a90003520 con 0x7f1aa810a590 2024-09-16T08:46:21.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f1a9c001580 con 0x7f1aa810a590 2024-09-16T08:46:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3749647368 0 0) 0x7f1aa81aab30 con 0x7f1aa8101a20 2024-09-16T08:46:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 >> v1:172.21.15.160:6790/0 conn(0x7f1aa810a590 legacy=0x7f1aa81a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 >> v1:172.21.15.169:6789/0 conn(0x7f1aa810e130 legacy=0x7f1aa81a80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa81acfd0 con 0x7f1aa8101a20 2024-09-16T08:46:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aad9b6640 1 -- 172.21.15.160:0/149761308 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa81abf60 con 0x7f1aa8101a20 2024-09-16T08:46:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.254+0000 7f1aad9b6640 1 -- 172.21.15.160:0/149761308 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f1aa81ac590 con 0x7f1aa8101a20 2024-09-16T08:46:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.255+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f1a98003a60 con 0x7f1aa8101a20 2024-09-16T08:46:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.255+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f1a98005b60 con 0x7f1aa8101a20 2024-09-16T08:46:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.255+0000 7f1aad9b6640 1 -- 172.21.15.160:0/149761308 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a94005080 con 0x7f1aa8101a20 2024-09-16T08:46:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.256+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7f1a980035f0 con 0x7f1aa8101a20 2024-09-16T08:46:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.257+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(64..64 src has 1..64) ==== 6172+0+0 (unknown 440296652 0 0) 0x7f1a980991d0 con 0x7f1aa8101a20 2024-09-16T08:46:21.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.261+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f1a980647e0 con 0x7f1aa8101a20 2024-09-16T08:46:21.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.436+0000 7f1aad9b6640 1 -- 172.21.15.160:0/149761308 --> v1:172.21.15.160:6800/2859204856 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a94000b90 con 0x7f1a9007bb30 2024-09-16T08:46:21.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.441+0000 7f1aac9b4640 1 -- 172.21.15.160:0/149761308 <== mgr.34526 v1:172.21.15.160:6800/2859204856 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+346453 (unknown 2965378022 0 774591111) 0x7f1a94000b90 con 0x7f1a9007bb30 2024-09-16T08:46:21.445 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:46:21.447 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-09-16T08:46:21.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.444+0000 7f1aad9b6640 1 -- 172.21.15.160:0/149761308 >> v1:172.21.15.160:6800/2859204856 conn(0x7f1a9007bb30 legacy=0x7f1a9007dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:21.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.444+0000 7f1aad9b6640 1 -- 172.21.15.160:0/149761308 >> v1:172.21.15.160:6789/0 conn(0x7f1aa8101a20 legacy=0x7f1aa810d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:21.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.444+0000 7f1aad9b6640 1 -- 172.21.15.160:0/149761308 shutdown_connections 2024-09-16T08:46:21.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.444+0000 7f1aad9b6640 1 -- 172.21.15.160:0/149761308 >> 172.21.15.160:0/149761308 conn(0x7f1aa80fd4b0 msgr2=0x7f1aa8107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:21.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.444+0000 7f1aad9b6640 1 -- 172.21.15.160:0/149761308 shutdown_connections 2024-09-16T08:46:21.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:21.444+0000 7f1aad9b6640 1 -- 172.21.15.160:0/149761308 wait complete. 2024-09-16T08:46:22.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:22 smithi160 ceph-mon[30137]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:22.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:22 smithi160 ceph-mon[30137]: from='client.14793 v1:172.21.15.160:0/149761308' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:46:22.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:22 smithi160 ceph-mon[40028]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:22.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:22 smithi160 ceph-mon[40028]: from='client.14793 v1:172.21.15.160:0/149761308' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:46:22.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:22 smithi169 ceph-mon[33465]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:22.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:22 smithi169 ceph-mon[33465]: from='client.14793 v1:172.21.15.160:0/149761308' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:46:22.891 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-09-16T08:46:21.040375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":888,"num_read_kb":745,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":239668,"kb_used_data":6812,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":749721548,"statfs":{"total":767960285184,"available":767714865152,"internally_reserved":0,"allocated":6975488,"data_stored":3721177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002875"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.024534+0000","last_change":"2024-09-16T08:45:33.813300+0000","last_active":"2024-09-16T08:46:03.024534+0000","last_peered":"2024-09-16T08:46:03.024534+0000","last_clean":"2024-09-16T08:46:03.024534+0000","last_became_active":"2024-09-16T08:45:33.813044+0000","last_became_peered":"2024-09-16T08:45:33.813044+0000","last_unstale":"2024-09-16T08:46:03.024534+0000","last_undegraded":"2024-09-16T08:46:03.024534+0000","last_fullsized":"2024-09-16T08:46:03.024534+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:14:51.205142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019302+0000","last_change":"2024-09-16T08:45:35.835577+0000","last_active":"2024-09-16T08:46:03.019302+0000","last_peered":"2024-09-16T08:46:03.019302+0000","last_clean":"2024-09-16T08:46:03.019302+0000","last_became_active":"2024-09-16T08:45:35.835280+0000","last_became_peered":"2024-09-16T08:45:35.835280+0000","last_unstale":"2024-09-16T08:46:03.019302+0000","last_undegraded":"2024-09-16T08:46:03.019302+0000","last_fullsized":"2024-09-16T08:46:03.019302+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:21:59.670525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.346018+0000","last_change":"2024-09-16T08:45:37.851433+0000","last_active":"2024-09-16T08:46:03.346018+0000","last_peered":"2024-09-16T08:46:03.346018+0000","last_clean":"2024-09-16T08:46:03.346018+0000","last_became_active":"2024-09-16T08:45:37.846008+0000","last_became_peered":"2024-09-16T08:45:37.846008+0000","last_unstale":"2024-09-16T08:46:03.346018+0000","last_undegraded":"2024-09-16T08:46:03.346018+0000","last_fullsized":"2024-09-16T08:46:03.346018+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:24:50.563483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019361+0000","last_change":"2024-09-16T08:45:39.869890+0000","last_active":"2024-09-16T08:46:03.019361+0000","last_peered":"2024-09-16T08:46:03.019361+0000","last_clean":"2024-09-16T08:46:03.019361+0000","last_became_active":"2024-09-16T08:45:39.869531+0000","last_became_peered":"2024-09-16T08:45:39.869531+0000","last_unstale":"2024-09-16T08:46:03.019361+0000","last_undegraded":"2024-09-16T08:46:03.019361+0000","last_fullsized":"2024-09-16T08:46:03.019361+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:25:31.737119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864425+0000","last_change":"2024-09-16T08:45:39.867250+0000","last_active":"2024-09-16T08:46:03.864425+0000","last_peered":"2024-09-16T08:46:03.864425+0000","last_clean":"2024-09-16T08:46:03.864425+0000","last_became_active":"2024-09-16T08:45:39.866898+0000","last_became_peered":"2024-09-16T08:45:39.866898+0000","last_unstale":"2024-09-16T08:46:03.864425+0000","last_undegraded":"2024-09-16T08:46:03.864425+0000","last_fullsized":"2024-09-16T08:46:03.864425+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:50:50.235631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.865696+0000","last_change":"2024-09-16T08:45:33.812271+0000","last_active":"2024-09-16T08:46:03.865696+0000","last_peered":"2024-09-16T08:46:03.865696+0000","last_clean":"2024-09-16T08:46:03.865696+0000","last_became_active":"2024-09-16T08:45:33.811993+0000","last_became_peered":"2024-09-16T08:45:33.811993+0000","last_unstale":"2024-09-16T08:46:03.865696+0000","last_undegraded":"2024-09-16T08:46:03.865696+0000","last_fullsized":"2024-09-16T08:46:03.865696+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:35.687926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"62'15","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.865999+0000","last_change":"2024-09-16T08:45:35.829196+0000","last_active":"2024-09-16T08:46:03.865999+0000","last_peered":"2024-09-16T08:46:03.865999+0000","last_clean":"2024-09-16T08:46:03.865999+0000","last_became_active":"2024-09-16T08:45:35.827938+0000","last_became_peered":"2024-09-16T08:45:35.827938+0000","last_unstale":"2024-09-16T08:46:03.865999+0000","last_undegraded":"2024-09-16T08:46:03.865999+0000","last_fullsized":"2024-09-16T08:46:03.865999+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:07.260344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019299+0000","last_change":"2024-09-16T08:45:37.846028+0000","last_active":"2024-09-16T08:46:03.019299+0000","last_peered":"2024-09-16T08:46:03.019299+0000","last_clean":"2024-09-16T08:46:03.019299+0000","last_became_active":"2024-09-16T08:45:37.845112+0000","last_became_peered":"2024-09-16T08:45:37.845112+0000","last_unstale":"2024-09-16T08:46:03.019299+0000","last_undegraded":"2024-09-16T08:46:03.019299+0000","last_fullsized":"2024-09-16T08:46:03.019299+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:04.613475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362612+0000","last_change":"2024-09-16T08:45:33.817292+0000","last_active":"2024-09-16T08:46:03.362612+0000","last_peered":"2024-09-16T08:46:03.362612+0000","last_clean":"2024-09-16T08:46:03.362612+0000","last_became_active":"2024-09-16T08:45:33.817001+0000","last_became_peered":"2024-09-16T08:45:33.817001+0000","last_unstale":"2024-09-16T08:46:03.362612+0000","last_undegraded":"2024-09-16T08:46:03.362612+0000","last_fullsized":"2024-09-16T08:46:03.362612+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:17.614320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019593+0000","last_change":"2024-09-16T08:45:35.826899+0000","last_active":"2024-09-16T08:46:03.019593+0000","last_peered":"2024-09-16T08:46:03.019593+0000","last_clean":"2024-09-16T08:46:03.019593+0000","last_became_active":"2024-09-16T08:45:35.826592+0000","last_became_peered":"2024-09-16T08:45:35.826592+0000","last_unstale":"2024-09-16T08:46:03.019593+0000","last_undegraded":"2024-09-16T08:46:03.019593+0000","last_fullsized":"2024-09-16T08:46:03.019593+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:34.217603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362253+0000","last_change":"2024-09-16T08:45:37.848960+0000","last_active":"2024-09-16T08:46:03.362253+0000","last_peered":"2024-09-16T08:46:03.362253+0000","last_clean":"2024-09-16T08:46:03.362253+0000","last_became_active":"2024-09-16T08:45:37.848542+0000","last_became_peered":"2024-09-16T08:45:37.848542+0000","last_unstale":"2024-09-16T08:46:03.362253+0000","last_undegraded":"2024-09-16T08:46:03.362253+0000","last_fullsized":"2024-09-16T08:46:03.362253+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:29.874057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019299+0000","last_change":"2024-09-16T08:45:39.865646+0000","last_active":"2024-09-16T08:46:03.019299+0000","last_peered":"2024-09-16T08:46:03.019299+0000","last_clean":"2024-09-16T08:46:03.019299+0000","last_became_active":"2024-09-16T08:45:39.864665+0000","last_became_peered":"2024-09-16T08:45:39.864665+0000","last_unstale":"2024-09-16T08:46:03.019299+0000","last_undegraded":"2024-09-16T08:46:03.019299+0000","last_fullsized":"2024-09-16T08:46:03.019299+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:25.934182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362345+0000","last_change":"2024-09-16T08:45:33.819294+0000","last_active":"2024-09-16T08:46:03.362345+0000","last_peered":"2024-09-16T08:46:03.362345+0000","last_clean":"2024-09-16T08:46:03.362345+0000","last_became_active":"2024-09-16T08:45:33.818672+0000","last_became_peered":"2024-09-16T08:45:33.818672+0000","last_unstale":"2024-09-16T08:46:03.362345+0000","last_undegraded":"2024-09-16T08:46:03.362345+0000","last_fullsized":"2024-09-16T08:46:03.362345+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:22:56.714569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"62'5","reported_seq":36,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020271+0000","last_change":"2024-09-16T08:45:35.835638+0000","last_active":"2024-09-16T08:46:03.020271+0000","last_peered":"2024-09-16T08:46:03.020271+0000","last_clean":"2024-09-16T08:46:03.020271+0000","last_became_active":"2024-09-16T08:45:35.835328+0000","last_became_peered":"2024-09-16T08:45:35.835328+0000","last_unstale":"2024-09-16T08:46:03.020271+0000","last_undegraded":"2024-09-16T08:46:03.020271+0000","last_fullsized":"2024-09-16T08:46:03.020271+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:22.259226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.367730+0000","last_change":"2024-09-16T08:45:37.846525+0000","last_active":"2024-09-16T08:46:03.367730+0000","last_peered":"2024-09-16T08:46:03.367730+0000","last_clean":"2024-09-16T08:46:03.367730+0000","last_became_active":"2024-09-16T08:45:37.846221+0000","last_became_peered":"2024-09-16T08:45:37.846221+0000","last_unstale":"2024-09-16T08:46:03.367730+0000","last_undegraded":"2024-09-16T08:46:03.367730+0000","last_fullsized":"2024-09-16T08:46:03.367730+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:20:38.234799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362070+0000","last_change":"2024-09-16T08:45:39.872162+0000","last_active":"2024-09-16T08:46:03.362070+0000","last_peered":"2024-09-16T08:46:03.362070+0000","last_clean":"2024-09-16T08:46:03.362070+0000","last_became_active":"2024-09-16T08:45:39.871888+0000","last_became_peered":"2024-09-16T08:45:39.871888+0000","last_unstale":"2024-09-16T08:46:03.362070+0000","last_undegraded":"2024-09-16T08:46:03.362070+0000","last_fullsized":"2024-09-16T08:46:03.362070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:15:43.600755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019856+0000","last_change":"2024-09-16T08:45:39.868502+0000","last_active":"2024-09-16T08:46:03.019856+0000","last_peered":"2024-09-16T08:46:03.019856+0000","last_clean":"2024-09-16T08:46:03.019856+0000","last_became_active":"2024-09-16T08:45:39.868192+0000","last_became_peered":"2024-09-16T08:45:39.868192+0000","last_unstale":"2024-09-16T08:46:03.019856+0000","last_undegraded":"2024-09-16T08:46:03.019856+0000","last_fullsized":"2024-09-16T08:46:03.019856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:10.995950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.024259+0000","last_change":"2024-09-16T08:45:33.811109+0000","last_active":"2024-09-16T08:46:03.024259+0000","last_peered":"2024-09-16T08:46:03.024259+0000","last_clean":"2024-09-16T08:46:03.024259+0000","last_became_active":"2024-09-16T08:45:33.810826+0000","last_became_peered":"2024-09-16T08:45:33.810826+0000","last_unstale":"2024-09-16T08:46:03.024259+0000","last_undegraded":"2024-09-16T08:46:03.024259+0000","last_fullsized":"2024-09-16T08:46:03.024259+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:42.551767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"62'15","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.022687+0000","last_change":"2024-09-16T08:45:35.833831+0000","last_active":"2024-09-16T08:46:03.022687+0000","last_peered":"2024-09-16T08:46:03.022687+0000","last_clean":"2024-09-16T08:46:03.022687+0000","last_became_active":"2024-09-16T08:45:35.833285+0000","last_became_peered":"2024-09-16T08:45:35.833285+0000","last_unstale":"2024-09-16T08:46:03.022687+0000","last_undegraded":"2024-09-16T08:46:03.022687+0000","last_fullsized":"2024-09-16T08:46:03.022687+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:56:10.339005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.346556+0000","last_change":"2024-09-16T08:45:37.853870+0000","last_active":"2024-09-16T08:46:03.346556+0000","last_peered":"2024-09-16T08:46:03.346556+0000","last_clean":"2024-09-16T08:46:03.346556+0000","last_became_active":"2024-09-16T08:45:37.852373+0000","last_became_peered":"2024-09-16T08:45:37.852373+0000","last_unstale":"2024-09-16T08:46:03.346556+0000","last_undegraded":"2024-09-16T08:46:03.346556+0000","last_fullsized":"2024-09-16T08:46:03.346556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:20.086294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.865363+0000","last_change":"2024-09-16T08:45:39.870832+0000","last_active":"2024-09-16T08:46:03.865363+0000","last_peered":"2024-09-16T08:46:03.865363+0000","last_clean":"2024-09-16T08:46:03.865363+0000","last_became_active":"2024-09-16T08:45:39.870528+0000","last_became_peered":"2024-09-16T08:45:39.870528+0000","last_unstale":"2024-09-16T08:46:03.865363+0000","last_undegraded":"2024-09-16T08:46:03.865363+0000","last_fullsized":"2024-09-16T08:46:03.865363+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:55.564948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019797+0000","last_change":"2024-09-16T08:45:33.812741+0000","last_active":"2024-09-16T08:46:03.019797+0000","last_peered":"2024-09-16T08:46:03.019797+0000","last_clean":"2024-09-16T08:46:03.019797+0000","last_became_active":"2024-09-16T08:45:33.812465+0000","last_became_peered":"2024-09-16T08:45:33.812465+0000","last_unstale":"2024-09-16T08:46:03.019797+0000","last_undegraded":"2024-09-16T08:46:03.019797+0000","last_fullsized":"2024-09-16T08:46:03.019797+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:18:41.804184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"62'12","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.865310+0000","last_change":"2024-09-16T08:45:35.835095+0000","last_active":"2024-09-16T08:46:03.865310+0000","last_peered":"2024-09-16T08:46:03.865310+0000","last_clean":"2024-09-16T08:46:03.865310+0000","last_became_active":"2024-09-16T08:45:35.834753+0000","last_became_peered":"2024-09-16T08:45:35.834753+0000","last_unstale":"2024-09-16T08:46:03.865310+0000","last_undegraded":"2024-09-16T08:46:03.865310+0000","last_fullsized":"2024-09-16T08:46:03.865310+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:29.974141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020302+0000","last_change":"2024-09-16T08:45:37.852729+0000","last_active":"2024-09-16T08:46:03.020302+0000","last_peered":"2024-09-16T08:46:03.020302+0000","last_clean":"2024-09-16T08:46:03.020302+0000","last_became_active":"2024-09-16T08:45:37.852416+0000","last_became_peered":"2024-09-16T08:45:37.852416+0000","last_unstale":"2024-09-16T08:46:03.020302+0000","last_undegraded":"2024-09-16T08:46:03.020302+0000","last_fullsized":"2024-09-16T08:46:03.020302+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:39.691495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"62'1","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.367981+0000","last_change":"2024-09-16T08:45:39.866127+0000","last_active":"2024-09-16T08:46:03.367981+0000","last_peered":"2024-09-16T08:46:03.367981+0000","last_clean":"2024-09-16T08:46:03.367981+0000","last_became_active":"2024-09-16T08:45:39.865860+0000","last_became_peered":"2024-09-16T08:45:39.865860+0000","last_unstale":"2024-09-16T08:46:03.367981+0000","last_undegraded":"2024-09-16T08:46:03.367981+0000","last_fullsized":"2024-09-16T08:46:03.367981+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:39.148052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"55'1","reported_seq":31,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.345584+0000","last_change":"2024-09-16T08:45:33.811044+0000","last_active":"2024-09-16T08:46:03.345584+0000","last_peered":"2024-09-16T08:46:03.345584+0000","last_clean":"2024-09-16T08:46:03.345584+0000","last_became_active":"2024-09-16T08:45:33.810752+0000","last_became_peered":"2024-09-16T08:45:33.810752+0000","last_unstale":"2024-09-16T08:46:03.345584+0000","last_undegraded":"2024-09-16T08:46:03.345584+0000","last_fullsized":"2024-09-16T08:46:03.345584+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:00:54.360250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"62'10","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020295+0000","last_change":"2024-09-16T08:45:35.841723+0000","last_active":"2024-09-16T08:46:03.020295+0000","last_peered":"2024-09-16T08:46:03.020295+0000","last_clean":"2024-09-16T08:46:03.020295+0000","last_became_active":"2024-09-16T08:45:35.841366+0000","last_became_peered":"2024-09-16T08:45:35.841366+0000","last_unstale":"2024-09-16T08:46:03.020295+0000","last_undegraded":"2024-09-16T08:46:03.020295+0000","last_fullsized":"2024-09-16T08:46:03.020295+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:00:35.472570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"62'8","reported_seq":43,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.865170+0000","last_change":"2024-09-16T08:45:37.849929+0000","last_active":"2024-09-16T08:46:13.865170+0000","last_peered":"2024-09-16T08:46:13.865170+0000","last_clean":"2024-09-16T08:46:13.865170+0000","last_became_active":"2024-09-16T08:45:37.849092+0000","last_became_peered":"2024-09-16T08:45:37.849092+0000","last_unstale":"2024-09-16T08:46:13.865170+0000","last_undegraded":"2024-09-16T08:46:13.865170+0000","last_fullsized":"2024-09-16T08:46:13.865170+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:17:04.338511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"62'15","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.346627+0000","last_change":"2024-09-16T08:45:35.838344+0000","last_active":"2024-09-16T08:46:03.346627+0000","last_peered":"2024-09-16T08:46:03.346627+0000","last_clean":"2024-09-16T08:46:03.346627+0000","last_became_active":"2024-09-16T08:45:35.838037+0000","last_became_peered":"2024-09-16T08:45:35.838037+0000","last_unstale":"2024-09-16T08:46:03.346627+0000","last_undegraded":"2024-09-16T08:46:03.346627+0000","last_fullsized":"2024-09-16T08:46:03.346627+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:12:53.229643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864415+0000","last_change":"2024-09-16T08:45:33.809967+0000","last_active":"2024-09-16T08:46:03.864415+0000","last_peered":"2024-09-16T08:46:03.864415+0000","last_clean":"2024-09-16T08:46:03.864415+0000","last_became_active":"2024-09-16T08:45:33.809640+0000","last_became_peered":"2024-09-16T08:45:33.809640+0000","last_unstale":"2024-09-16T08:46:03.864415+0000","last_undegraded":"2024-09-16T08:46:03.864415+0000","last_fullsized":"2024-09-16T08:46:03.864415+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:06.774145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"62'8","reported_seq":40,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.865445+0000","last_change":"2024-09-16T08:45:37.845825+0000","last_active":"2024-09-16T08:46:13.865445+0000","last_peered":"2024-09-16T08:46:13.865445+0000","last_clean":"2024-09-16T08:46:13.865445+0000","last_became_active":"2024-09-16T08:45:37.845541+0000","last_became_peered":"2024-09-16T08:45:37.845541+0000","last_unstale":"2024-09-16T08:46:13.865445+0000","last_undegraded":"2024-09-16T08:46:13.865445+0000","last_fullsized":"2024-09-16T08:46:13.865445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:58:25.792250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362358+0000","last_change":"2024-09-16T08:45:39.871072+0000","last_active":"2024-09-16T08:46:03.362358+0000","last_peered":"2024-09-16T08:46:03.362358+0000","last_clean":"2024-09-16T08:46:03.362358+0000","last_became_active":"2024-09-16T08:45:39.870793+0000","last_became_peered":"2024-09-16T08:45:39.870793+0000","last_unstale":"2024-09-16T08:46:03.362358+0000","last_undegraded":"2024-09-16T08:46:03.362358+0000","last_fullsized":"2024-09-16T08:46:03.362358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:18:06.743025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"62'18","reported_seq":58,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864217+0000","last_change":"2024-09-16T08:45:35.840066+0000","last_active":"2024-09-16T08:46:03.864217+0000","last_peered":"2024-09-16T08:46:03.864217+0000","last_clean":"2024-09-16T08:46:03.864217+0000","last_became_active":"2024-09-16T08:45:35.839601+0000","last_became_peered":"2024-09-16T08:45:35.839601+0000","last_unstale":"2024-09-16T08:46:03.864217+0000","last_undegraded":"2024-09-16T08:46:03.864217+0000","last_fullsized":"2024-09-16T08:46:03.864217+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:04.727428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.865950+0000","last_change":"2024-09-16T08:45:33.811448+0000","last_active":"2024-09-16T08:46:03.865950+0000","last_peered":"2024-09-16T08:46:03.865950+0000","last_clean":"2024-09-16T08:46:03.865950+0000","last_became_active":"2024-09-16T08:45:33.811187+0000","last_became_peered":"2024-09-16T08:45:33.811187+0000","last_unstale":"2024-09-16T08:46:03.865950+0000","last_undegraded":"2024-09-16T08:46:03.865950+0000","last_fullsized":"2024-09-16T08:46:03.865950+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:41:41.933623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019349+0000","last_change":"2024-09-16T08:45:37.851147+0000","last_active":"2024-09-16T08:46:03.019349+0000","last_peered":"2024-09-16T08:46:03.019349+0000","last_clean":"2024-09-16T08:46:03.019349+0000","last_became_active":"2024-09-16T08:45:37.850717+0000","last_became_peered":"2024-09-16T08:45:37.850717+0000","last_unstale":"2024-09-16T08:46:03.019349+0000","last_undegraded":"2024-09-16T08:46:03.019349+0000","last_fullsized":"2024-09-16T08:46:03.019349+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:31:35.555632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019566+0000","last_change":"2024-09-16T08:45:39.867089+0000","last_active":"2024-09-16T08:46:03.019566+0000","last_peered":"2024-09-16T08:46:03.019566+0000","last_clean":"2024-09-16T08:46:03.019566+0000","last_became_active":"2024-09-16T08:45:39.866784+0000","last_became_peered":"2024-09-16T08:45:39.866784+0000","last_unstale":"2024-09-16T08:46:03.019566+0000","last_undegraded":"2024-09-16T08:46:03.019566+0000","last_fullsized":"2024-09-16T08:46:03.019566+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:47.876056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"62'14","reported_seq":47,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019358+0000","last_change":"2024-09-16T08:45:35.826507+0000","last_active":"2024-09-16T08:46:03.019358+0000","last_peered":"2024-09-16T08:46:03.019358+0000","last_clean":"2024-09-16T08:46:03.019358+0000","last_became_active":"2024-09-16T08:45:35.826160+0000","last_became_peered":"2024-09-16T08:45:35.826160+0000","last_unstale":"2024-09-16T08:46:03.019358+0000","last_undegraded":"2024-09-16T08:46:03.019358+0000","last_fullsized":"2024-09-16T08:46:03.019358+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:55:24.026157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"55'1","reported_seq":31,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.021062+0000","last_change":"2024-09-16T08:45:33.808737+0000","last_active":"2024-09-16T08:46:03.021062+0000","last_peered":"2024-09-16T08:46:03.021062+0000","last_clean":"2024-09-16T08:46:03.021062+0000","last_became_active":"2024-09-16T08:45:33.808424+0000","last_became_peered":"2024-09-16T08:45:33.808424+0000","last_unstale":"2024-09-16T08:46:03.021062+0000","last_undegraded":"2024-09-16T08:46:03.021062+0000","last_fullsized":"2024-09-16T08:46:03.021062+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:41.655775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"62'8","reported_seq":40,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.864959+0000","last_change":"2024-09-16T08:45:37.855366+0000","last_active":"2024-09-16T08:46:13.864959+0000","last_peered":"2024-09-16T08:46:13.864959+0000","last_clean":"2024-09-16T08:46:13.864959+0000","last_became_active":"2024-09-16T08:45:37.855008+0000","last_became_peered":"2024-09-16T08:45:37.855008+0000","last_unstale":"2024-09-16T08:46:13.864959+0000","last_undegraded":"2024-09-16T08:46:13.864959+0000","last_fullsized":"2024-09-16T08:46:13.864959+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:32.170198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.367835+0000","last_change":"2024-09-16T08:45:39.866879+0000","last_active":"2024-09-16T08:46:03.367835+0000","last_peered":"2024-09-16T08:46:03.367835+0000","last_clean":"2024-09-16T08:46:03.367835+0000","last_became_active":"2024-09-16T08:45:39.866594+0000","last_became_peered":"2024-09-16T08:45:39.866594+0000","last_unstale":"2024-09-16T08:46:03.367835+0000","last_undegraded":"2024-09-16T08:46:03.367835+0000","last_fullsized":"2024-09-16T08:46:03.367835+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:16:39.764595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"62'10","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.345776+0000","last_change":"2024-09-16T08:45:35.835159+0000","last_active":"2024-09-16T08:46:03.345776+0000","last_peered":"2024-09-16T08:46:03.345776+0000","last_clean":"2024-09-16T08:46:03.345776+0000","last_became_active":"2024-09-16T08:45:35.833871+0000","last_became_peered":"2024-09-16T08:45:35.833871+0000","last_unstale":"2024-09-16T08:46:03.345776+0000","last_undegraded":"2024-09-16T08:46:03.345776+0000","last_fullsized":"2024-09-16T08:46:03.345776+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:37:29.528117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362691+0000","last_change":"2024-09-16T08:45:33.817698+0000","last_active":"2024-09-16T08:46:03.362691+0000","last_peered":"2024-09-16T08:46:03.362691+0000","last_clean":"2024-09-16T08:46:03.362691+0000","last_became_active":"2024-09-16T08:45:33.817381+0000","last_became_peered":"2024-09-16T08:45:33.817381+0000","last_unstale":"2024-09-16T08:46:03.362691+0000","last_undegraded":"2024-09-16T08:46:03.362691+0000","last_fullsized":"2024-09-16T08:46:03.362691+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:11:54.619759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"62'8","reported_seq":40,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.864970+0000","last_change":"2024-09-16T08:45:37.849922+0000","last_active":"2024-09-16T08:46:13.864970+0000","last_peered":"2024-09-16T08:46:13.864970+0000","last_clean":"2024-09-16T08:46:13.864970+0000","last_became_active":"2024-09-16T08:45:37.849621+0000","last_became_peered":"2024-09-16T08:45:37.849621+0000","last_unstale":"2024-09-16T08:46:13.864970+0000","last_undegraded":"2024-09-16T08:46:13.864970+0000","last_fullsized":"2024-09-16T08:46:13.864970+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:21:07.076075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019444+0000","last_change":"2024-09-16T08:45:39.864584+0000","last_active":"2024-09-16T08:46:03.019444+0000","last_peered":"2024-09-16T08:46:03.019444+0000","last_clean":"2024-09-16T08:46:03.019444+0000","last_became_active":"2024-09-16T08:45:39.863463+0000","last_became_peered":"2024-09-16T08:45:39.863463+0000","last_unstale":"2024-09-16T08:46:03.019444+0000","last_undegraded":"2024-09-16T08:46:03.019444+0000","last_fullsized":"2024-09-16T08:46:03.019444+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:27.795309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"62'19","reported_seq":62,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.023975+0000","last_change":"2024-09-16T08:45:35.838887+0000","last_active":"2024-09-16T08:46:03.023975+0000","last_peered":"2024-09-16T08:46:03.023975+0000","last_clean":"2024-09-16T08:46:03.023975+0000","last_became_active":"2024-09-16T08:45:35.838565+0000","last_became_peered":"2024-09-16T08:45:35.838565+0000","last_unstale":"2024-09-16T08:46:03.023975+0000","last_undegraded":"2024-09-16T08:46:03.023975+0000","last_fullsized":"2024-09-16T08:46:03.023975+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:51.681132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.346247+0000","last_change":"2024-09-16T08:45:33.812693+0000","last_active":"2024-09-16T08:46:03.346247+0000","last_peered":"2024-09-16T08:46:03.346247+0000","last_clean":"2024-09-16T08:46:03.346247+0000","last_became_active":"2024-09-16T08:45:33.811644+0000","last_became_peered":"2024-09-16T08:45:33.811644+0000","last_unstale":"2024-09-16T08:46:03.346247+0000","last_undegraded":"2024-09-16T08:46:03.346247+0000","last_fullsized":"2024-09-16T08:46:03.346247+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:45:46.664593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.412555+0000","last_change":"2024-09-16T08:45:37.848576+0000","last_active":"2024-09-16T08:46:03.412555+0000","last_peered":"2024-09-16T08:46:03.412555+0000","last_clean":"2024-09-16T08:46:03.412555+0000","last_became_active":"2024-09-16T08:45:37.848298+0000","last_became_peered":"2024-09-16T08:45:37.848298+0000","last_unstale":"2024-09-16T08:46:03.412555+0000","last_undegraded":"2024-09-16T08:46:03.412555+0000","last_fullsized":"2024-09-16T08:46:03.412555+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:09:18.833845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.345407+0000","last_change":"2024-09-16T08:45:39.866848+0000","last_active":"2024-09-16T08:46:03.345407+0000","last_peered":"2024-09-16T08:46:03.345407+0000","last_clean":"2024-09-16T08:46:03.345407+0000","last_became_active":"2024-09-16T08:45:39.866559+0000","last_became_peered":"2024-09-16T08:45:39.866559+0000","last_unstale":"2024-09-16T08:46:03.345407+0000","last_undegraded":"2024-09-16T08:46:03.345407+0000","last_fullsized":"2024-09-16T08:46:03.345407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:19:44.414025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"62'28","reported_seq":84,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.864959+0000","last_change":"2024-09-16T08:45:35.832443+0000","last_active":"2024-09-16T08:46:13.864959+0000","last_peered":"2024-09-16T08:46:13.864959+0000","last_clean":"2024-09-16T08:46:13.864959+0000","last_became_active":"2024-09-16T08:45:35.832110+0000","last_became_peered":"2024-09-16T08:45:35.832110+0000","last_unstale":"2024-09-16T08:46:13.864959+0000","last_undegraded":"2024-09-16T08:46:13.864959+0000","last_fullsized":"2024-09-16T08:46:13.864959+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:16:55.691286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020650+0000","last_change":"2024-09-16T08:45:33.815029+0000","last_active":"2024-09-16T08:46:03.020650+0000","last_peered":"2024-09-16T08:46:03.020650+0000","last_clean":"2024-09-16T08:46:03.020650+0000","last_became_active":"2024-09-16T08:45:33.814734+0000","last_became_peered":"2024-09-16T08:45:33.814734+0000","last_unstale":"2024-09-16T08:46:03.020650+0000","last_undegraded":"2024-09-16T08:46:03.020650+0000","last_fullsized":"2024-09-16T08:46:03.020650+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:30.492050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"57'2","reported_seq":40,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362310+0000","last_change":"2024-09-16T08:45:35.824559+0000","last_active":"2024-09-16T08:46:03.362310+0000","last_peered":"2024-09-16T08:46:03.362310+0000","last_clean":"2024-09-16T08:46:03.362310+0000","last_became_active":"2024-09-16T08:45:32.804280+0000","last_became_peered":"2024-09-16T08:45:32.804280+0000","last_unstale":"2024-09-16T08:46:03.362310+0000","last_undegraded":"2024-09-16T08:46:03.362310+0000","last_fullsized":"2024-09-16T08:46:03.362310+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:31.787970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:31.787970+0000","last_clean_scrub_stamp":"2024-09-16T08:45:31.787970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:09:07.773521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001552082,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.023281+0000","last_change":"2024-09-16T08:45:37.845569+0000","last_active":"2024-09-16T08:46:03.023281+0000","last_peered":"2024-09-16T08:46:03.023281+0000","last_clean":"2024-09-16T08:46:03.023281+0000","last_became_active":"2024-09-16T08:45:37.845272+0000","last_became_peered":"2024-09-16T08:45:37.845272+0000","last_unstale":"2024-09-16T08:46:03.023281+0000","last_undegraded":"2024-09-16T08:46:03.023281+0000","last_fullsized":"2024-09-16T08:46:03.023281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:51.576180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"62'1","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.865338+0000","last_change":"2024-09-16T08:45:39.872662+0000","last_active":"2024-09-16T08:46:03.865338+0000","last_peered":"2024-09-16T08:46:03.865338+0000","last_clean":"2024-09-16T08:46:03.865338+0000","last_became_active":"2024-09-16T08:45:39.872337+0000","last_became_peered":"2024-09-16T08:45:39.872337+0000","last_unstale":"2024-09-16T08:46:03.865338+0000","last_undegraded":"2024-09-16T08:46:03.865338+0000","last_fullsized":"2024-09-16T08:46:03.865338+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:59.658604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"62'13","reported_seq":53,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.346751+0000","last_change":"2024-09-16T08:45:35.834310+0000","last_active":"2024-09-16T08:46:03.346751+0000","last_peered":"2024-09-16T08:46:03.346751+0000","last_clean":"2024-09-16T08:46:03.346751+0000","last_became_active":"2024-09-16T08:45:35.833790+0000","last_became_peered":"2024-09-16T08:45:35.833790+0000","last_unstale":"2024-09-16T08:46:03.346751+0000","last_undegraded":"2024-09-16T08:46:03.346751+0000","last_fullsized":"2024-09-16T08:46:03.346751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:38:14.928129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.345506+0000","last_change":"2024-09-16T08:45:33.813084+0000","last_active":"2024-09-16T08:46:03.345506+0000","last_peered":"2024-09-16T08:46:03.345506+0000","last_clean":"2024-09-16T08:46:03.345506+0000","last_became_active":"2024-09-16T08:45:33.812820+0000","last_became_peered":"2024-09-16T08:45:33.812820+0000","last_unstale":"2024-09-16T08:46:03.345506+0000","last_undegraded":"2024-09-16T08:46:03.345506+0000","last_fullsized":"2024-09-16T08:46:03.345506+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:54.637276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"55'1","reported_seq":39,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.021750+0000","last_change":"2024-09-16T08:45:35.818992+0000","last_active":"2024-09-16T08:46:03.021750+0000","last_peered":"2024-09-16T08:46:03.021750+0000","last_clean":"2024-09-16T08:46:03.021750+0000","last_became_active":"2024-09-16T08:45:32.802160+0000","last_became_peered":"2024-09-16T08:45:32.802160+0000","last_unstale":"2024-09-16T08:46:03.021750+0000","last_undegraded":"2024-09-16T08:46:03.021750+0000","last_fullsized":"2024-09-16T08:46:03.021750+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:31.787970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:31.787970+0000","last_clean_scrub_stamp":"2024-09-16T08:45:31.787970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:40.625663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060482500000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.022319+0000","last_change":"2024-09-16T08:45:37.846890+0000","last_active":"2024-09-16T08:46:03.022319+0000","last_peered":"2024-09-16T08:46:03.022319+0000","last_clean":"2024-09-16T08:46:03.022319+0000","last_became_active":"2024-09-16T08:45:37.846582+0000","last_became_peered":"2024-09-16T08:45:37.846582+0000","last_unstale":"2024-09-16T08:46:03.022319+0000","last_undegraded":"2024-09-16T08:46:03.022319+0000","last_fullsized":"2024-09-16T08:46:03.022319+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:22.783666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.368113+0000","last_change":"2024-09-16T08:45:39.865107+0000","last_active":"2024-09-16T08:46:03.368113+0000","last_peered":"2024-09-16T08:46:03.368113+0000","last_clean":"2024-09-16T08:46:03.368113+0000","last_became_active":"2024-09-16T08:45:39.864836+0000","last_became_peered":"2024-09-16T08:45:39.864836+0000","last_unstale":"2024-09-16T08:46:03.368113+0000","last_undegraded":"2024-09-16T08:46:03.368113+0000","last_fullsized":"2024-09-16T08:46:03.368113+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:38.365579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"62'12","reported_seq":44,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020797+0000","last_change":"2024-09-16T08:45:35.834103+0000","last_active":"2024-09-16T08:46:03.020797+0000","last_peered":"2024-09-16T08:46:03.020797+0000","last_clean":"2024-09-16T08:46:03.020797+0000","last_became_active":"2024-09-16T08:45:35.833797+0000","last_became_peered":"2024-09-16T08:45:35.833797+0000","last_unstale":"2024-09-16T08:46:03.020797+0000","last_undegraded":"2024-09-16T08:46:03.020797+0000","last_fullsized":"2024-09-16T08:46:03.020797+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:00:11.203579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019424+0000","last_change":"2024-09-16T08:45:33.810293+0000","last_active":"2024-09-16T08:46:03.019424+0000","last_peered":"2024-09-16T08:46:03.019424+0000","last_clean":"2024-09-16T08:46:03.019424+0000","last_became_active":"2024-09-16T08:45:33.809995+0000","last_became_peered":"2024-09-16T08:45:33.809995+0000","last_unstale":"2024-09-16T08:46:03.019424+0000","last_undegraded":"2024-09-16T08:46:03.019424+0000","last_fullsized":"2024-09-16T08:46:03.019424+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:46:05.431730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":81,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:17.731660+0000","last_change":"2024-09-16T08:45:35.837001+0000","last_active":"2024-09-16T08:46:17.731660+0000","last_peered":"2024-09-16T08:46:17.731660+0000","last_clean":"2024-09-16T08:46:17.731660+0000","last_became_active":"2024-09-16T08:45:32.805284+0000","last_became_peered":"2024-09-16T08:45:32.805284+0000","last_unstale":"2024-09-16T08:46:17.731660+0000","last_undegraded":"2024-09-16T08:46:17.731660+0000","last_fullsized":"2024-09-16T08:46:17.731660+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:31.787970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:31.787970+0000","last_clean_scrub_stamp":"2024-09-16T08:45:31.787970+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:06:19.733606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093979999999999997,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":40,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362749+0000","last_change":"2024-09-16T08:45:37.853088+0000","last_active":"2024-09-16T08:46:03.362749+0000","last_peered":"2024-09-16T08:46:03.362749+0000","last_clean":"2024-09-16T08:46:03.362749+0000","last_became_active":"2024-09-16T08:45:37.852639+0000","last_became_peered":"2024-09-16T08:45:37.852639+0000","last_unstale":"2024-09-16T08:46:03.362749+0000","last_undegraded":"2024-09-16T08:46:03.362749+0000","last_fullsized":"2024-09-16T08:46:03.362749+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:54.021042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.345615+0000","last_change":"2024-09-16T08:45:39.869824+0000","last_active":"2024-09-16T08:46:03.345615+0000","last_peered":"2024-09-16T08:46:03.345615+0000","last_clean":"2024-09-16T08:46:03.345615+0000","last_became_active":"2024-09-16T08:45:39.869497+0000","last_became_peered":"2024-09-16T08:45:39.869497+0000","last_unstale":"2024-09-16T08:46:03.345615+0000","last_undegraded":"2024-09-16T08:46:03.345615+0000","last_fullsized":"2024-09-16T08:46:03.345615+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:49:47.403086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"62'16","reported_seq":58,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.865222+0000","last_change":"2024-09-16T08:45:35.839016+0000","last_active":"2024-09-16T08:46:13.865222+0000","last_peered":"2024-09-16T08:46:13.865222+0000","last_clean":"2024-09-16T08:46:13.865222+0000","last_became_active":"2024-09-16T08:45:35.838667+0000","last_became_peered":"2024-09-16T08:45:35.838667+0000","last_unstale":"2024-09-16T08:46:13.865222+0000","last_undegraded":"2024-09-16T08:46:13.865222+0000","last_fullsized":"2024-09-16T08:46:13.865222+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:52:20.141633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864591+0000","last_change":"2024-09-16T08:45:33.813160+0000","last_active":"2024-09-16T08:46:03.864591+0000","last_peered":"2024-09-16T08:46:03.864591+0000","last_clean":"2024-09-16T08:46:03.864591+0000","last_became_active":"2024-09-16T08:45:33.812868+0000","last_became_peered":"2024-09-16T08:45:33.812868+0000","last_unstale":"2024-09-16T08:46:03.864591+0000","last_undegraded":"2024-09-16T08:46:03.864591+0000","last_fullsized":"2024-09-16T08:46:03.864591+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:02.449848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"64'79","reported_seq":67,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:05.341213+0000","last_change":"2024-09-16T08:45:25.077647+0000","last_active":"2024-09-16T08:46:05.341213+0000","last_peered":"2024-09-16T08:46:05.341213+0000","last_clean":"2024-09-16T08:46:05.341213+0000","last_became_active":"2024-09-16T08:45:25.071716+0000","last_became_peered":"2024-09-16T08:45:25.071716+0000","last_unstale":"2024-09-16T08:46:05.341213+0000","last_undegraded":"2024-09-16T08:46:05.341213+0000","last_fullsized":"2024-09-16T08:46:05.341213+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:43:02.998358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:43:02.998358+0000","last_clean_scrub_stamp":"2024-09-16T08:43:02.998358+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:50.593602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.368058+0000","last_change":"2024-09-16T08:45:37.847599+0000","last_active":"2024-09-16T08:46:03.368058+0000","last_peered":"2024-09-16T08:46:03.368058+0000","last_clean":"2024-09-16T08:46:03.368058+0000","last_became_active":"2024-09-16T08:45:37.846998+0000","last_became_peered":"2024-09-16T08:45:37.846998+0000","last_unstale":"2024-09-16T08:46:03.368058+0000","last_undegraded":"2024-09-16T08:46:03.368058+0000","last_fullsized":"2024-09-16T08:46:03.368058+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:38.641395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362668+0000","last_change":"2024-09-16T08:45:39.872581+0000","last_active":"2024-09-16T08:46:03.362668+0000","last_peered":"2024-09-16T08:46:03.362668+0000","last_clean":"2024-09-16T08:46:03.362668+0000","last_became_active":"2024-09-16T08:45:39.871964+0000","last_became_peered":"2024-09-16T08:45:39.871964+0000","last_unstale":"2024-09-16T08:46:03.362668+0000","last_undegraded":"2024-09-16T08:46:03.362668+0000","last_fullsized":"2024-09-16T08:46:03.362668+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:21:10.767288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"62'11","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019944+0000","last_change":"2024-09-16T08:45:35.834626+0000","last_active":"2024-09-16T08:46:03.019944+0000","last_peered":"2024-09-16T08:46:03.019944+0000","last_clean":"2024-09-16T08:46:03.019944+0000","last_became_active":"2024-09-16T08:45:35.833959+0000","last_became_peered":"2024-09-16T08:45:35.833959+0000","last_unstale":"2024-09-16T08:46:03.019944+0000","last_undegraded":"2024-09-16T08:46:03.019944+0000","last_fullsized":"2024-09-16T08:46:03.019944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:10.384496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019601+0000","last_change":"2024-09-16T08:45:33.813379+0000","last_active":"2024-09-16T08:46:03.019601+0000","last_peered":"2024-09-16T08:46:03.019601+0000","last_clean":"2024-09-16T08:46:03.019601+0000","last_became_active":"2024-09-16T08:45:33.813115+0000","last_became_peered":"2024-09-16T08:45:33.813115+0000","last_unstale":"2024-09-16T08:46:03.019601+0000","last_undegraded":"2024-09-16T08:46:03.019601+0000","last_fullsized":"2024-09-16T08:46:03.019601+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:47:55.377999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362627+0000","last_change":"2024-09-16T08:45:37.851256+0000","last_active":"2024-09-16T08:46:03.362627+0000","last_peered":"2024-09-16T08:46:03.362627+0000","last_clean":"2024-09-16T08:46:03.362627+0000","last_became_active":"2024-09-16T08:45:37.850975+0000","last_became_peered":"2024-09-16T08:45:37.850975+0000","last_unstale":"2024-09-16T08:46:03.362627+0000","last_undegraded":"2024-09-16T08:46:03.362627+0000","last_fullsized":"2024-09-16T08:46:03.362627+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:28.209730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864414+0000","last_change":"2024-09-16T08:45:39.868671+0000","last_active":"2024-09-16T08:46:03.864414+0000","last_peered":"2024-09-16T08:46:03.864414+0000","last_clean":"2024-09-16T08:46:03.864414+0000","last_became_active":"2024-09-16T08:45:39.868367+0000","last_became_peered":"2024-09-16T08:45:39.868367+0000","last_unstale":"2024-09-16T08:46:03.864414+0000","last_undegraded":"2024-09-16T08:46:03.864414+0000","last_fullsized":"2024-09-16T08:46:03.864414+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:00:15.182398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"62'17","reported_seq":54,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020100+0000","last_change":"2024-09-16T08:45:35.833654+0000","last_active":"2024-09-16T08:46:03.020100+0000","last_peered":"2024-09-16T08:46:03.020100+0000","last_clean":"2024-09-16T08:46:03.020100+0000","last_became_active":"2024-09-16T08:45:35.832032+0000","last_became_peered":"2024-09-16T08:45:35.832032+0000","last_unstale":"2024-09-16T08:46:03.020100+0000","last_undegraded":"2024-09-16T08:46:03.020100+0000","last_fullsized":"2024-09-16T08:46:03.020100+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:40.179999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.412841+0000","last_change":"2024-09-16T08:45:33.814328+0000","last_active":"2024-09-16T08:46:03.412841+0000","last_peered":"2024-09-16T08:46:03.412841+0000","last_clean":"2024-09-16T08:46:03.412841+0000","last_became_active":"2024-09-16T08:45:33.814010+0000","last_became_peered":"2024-09-16T08:45:33.814010+0000","last_unstale":"2024-09-16T08:46:03.412841+0000","last_undegraded":"2024-09-16T08:46:03.412841+0000","last_fullsized":"2024-09-16T08:46:03.412841+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:53:02.457216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.345708+0000","last_change":"2024-09-16T08:45:37.854068+0000","last_active":"2024-09-16T08:46:03.345708+0000","last_peered":"2024-09-16T08:46:03.345708+0000","last_clean":"2024-09-16T08:46:03.345708+0000","last_became_active":"2024-09-16T08:45:37.853565+0000","last_became_peered":"2024-09-16T08:45:37.853565+0000","last_unstale":"2024-09-16T08:46:03.345708+0000","last_undegraded":"2024-09-16T08:46:03.345708+0000","last_fullsized":"2024-09-16T08:46:03.345708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:47.205588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.022507+0000","last_change":"2024-09-16T08:45:39.871548+0000","last_active":"2024-09-16T08:46:03.022507+0000","last_peered":"2024-09-16T08:46:03.022507+0000","last_clean":"2024-09-16T08:46:03.022507+0000","last_became_active":"2024-09-16T08:45:39.871231+0000","last_became_peered":"2024-09-16T08:45:39.871231+0000","last_unstale":"2024-09-16T08:46:03.022507+0000","last_undegraded":"2024-09-16T08:46:03.022507+0000","last_fullsized":"2024-09-16T08:46:03.022507+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:53.365040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"62'10","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020349+0000","last_change":"2024-09-16T08:45:35.828922+0000","last_active":"2024-09-16T08:46:03.020349+0000","last_peered":"2024-09-16T08:46:03.020349+0000","last_clean":"2024-09-16T08:46:03.020349+0000","last_became_active":"2024-09-16T08:45:35.828478+0000","last_became_peered":"2024-09-16T08:45:35.828478+0000","last_unstale":"2024-09-16T08:46:03.020349+0000","last_undegraded":"2024-09-16T08:46:03.020349+0000","last_fullsized":"2024-09-16T08:46:03.020349+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:51.679150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"55'1","reported_seq":36,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864829+0000","last_change":"2024-09-16T08:45:33.810637+0000","last_active":"2024-09-16T08:46:03.864829+0000","last_peered":"2024-09-16T08:46:03.864829+0000","last_clean":"2024-09-16T08:46:03.864829+0000","last_became_active":"2024-09-16T08:45:33.810348+0000","last_became_peered":"2024-09-16T08:45:33.810348+0000","last_unstale":"2024-09-16T08:46:03.864829+0000","last_undegraded":"2024-09-16T08:46:03.864829+0000","last_fullsized":"2024-09-16T08:46:03.864829+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:43:54.969984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"62'8","reported_seq":40,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.864970+0000","last_change":"2024-09-16T08:45:37.847357+0000","last_active":"2024-09-16T08:46:13.864970+0000","last_peered":"2024-09-16T08:46:13.864970+0000","last_clean":"2024-09-16T08:46:13.864970+0000","last_became_active":"2024-09-16T08:45:37.847049+0000","last_became_peered":"2024-09-16T08:45:37.847049+0000","last_unstale":"2024-09-16T08:46:13.864970+0000","last_undegraded":"2024-09-16T08:46:13.864970+0000","last_fullsized":"2024-09-16T08:46:13.864970+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:12:49.100003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019733+0000","last_change":"2024-09-16T08:45:39.869759+0000","last_active":"2024-09-16T08:46:03.019733+0000","last_peered":"2024-09-16T08:46:03.019733+0000","last_clean":"2024-09-16T08:46:03.019733+0000","last_became_active":"2024-09-16T08:45:39.869494+0000","last_became_peered":"2024-09-16T08:45:39.869494+0000","last_unstale":"2024-09-16T08:46:03.019733+0000","last_undegraded":"2024-09-16T08:46:03.019733+0000","last_fullsized":"2024-09-16T08:46:03.019733+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:18.189832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.023865+0000","last_change":"2024-09-16T08:45:35.833441+0000","last_active":"2024-09-16T08:46:03.023865+0000","last_peered":"2024-09-16T08:46:03.023865+0000","last_clean":"2024-09-16T08:46:03.023865+0000","last_became_active":"2024-09-16T08:45:35.833127+0000","last_became_peered":"2024-09-16T08:45:35.833127+0000","last_unstale":"2024-09-16T08:46:03.023865+0000","last_undegraded":"2024-09-16T08:46:03.023865+0000","last_fullsized":"2024-09-16T08:46:03.023865+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:45:35.228973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362406+0000","last_change":"2024-09-16T08:45:33.818979+0000","last_active":"2024-09-16T08:46:03.362406+0000","last_peered":"2024-09-16T08:46:03.362406+0000","last_clean":"2024-09-16T08:46:03.362406+0000","last_became_active":"2024-09-16T08:45:33.818114+0000","last_became_peered":"2024-09-16T08:45:33.818114+0000","last_unstale":"2024-09-16T08:46:03.362406+0000","last_undegraded":"2024-09-16T08:46:03.362406+0000","last_fullsized":"2024-09-16T08:46:03.362406+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:25.701180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.022092+0000","last_change":"2024-09-16T08:45:37.850266+0000","last_active":"2024-09-16T08:46:03.022092+0000","last_peered":"2024-09-16T08:46:03.022092+0000","last_clean":"2024-09-16T08:46:03.022092+0000","last_became_active":"2024-09-16T08:45:37.849966+0000","last_became_peered":"2024-09-16T08:45:37.849966+0000","last_unstale":"2024-09-16T08:46:03.022092+0000","last_undegraded":"2024-09-16T08:46:03.022092+0000","last_fullsized":"2024-09-16T08:46:03.022092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:23:21.782733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019266+0000","last_change":"2024-09-16T08:45:39.864769+0000","last_active":"2024-09-16T08:46:03.019266+0000","last_peered":"2024-09-16T08:46:03.019266+0000","last_clean":"2024-09-16T08:46:03.019266+0000","last_became_active":"2024-09-16T08:45:39.863928+0000","last_became_peered":"2024-09-16T08:45:39.863928+0000","last_unstale":"2024-09-16T08:46:03.019266+0000","last_undegraded":"2024-09-16T08:46:03.019266+0000","last_fullsized":"2024-09-16T08:46:03.019266+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:35.905425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"62'19","reported_seq":57,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.412658+0000","last_change":"2024-09-16T08:45:35.842013+0000","last_active":"2024-09-16T08:46:03.412658+0000","last_peered":"2024-09-16T08:46:03.412658+0000","last_clean":"2024-09-16T08:46:03.412658+0000","last_became_active":"2024-09-16T08:45:35.840965+0000","last_became_peered":"2024-09-16T08:45:35.840965+0000","last_unstale":"2024-09-16T08:46:03.412658+0000","last_undegraded":"2024-09-16T08:46:03.412658+0000","last_fullsized":"2024-09-16T08:46:03.412658+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:13.342411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.368427+0000","last_change":"2024-09-16T08:45:33.814098+0000","last_active":"2024-09-16T08:46:03.368427+0000","last_peered":"2024-09-16T08:46:03.368427+0000","last_clean":"2024-09-16T08:46:03.368427+0000","last_became_active":"2024-09-16T08:45:33.813825+0000","last_became_peered":"2024-09-16T08:45:33.813825+0000","last_unstale":"2024-09-16T08:46:03.368427+0000","last_undegraded":"2024-09-16T08:46:03.368427+0000","last_fullsized":"2024-09-16T08:46:03.368427+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:56:39.972190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.021932+0000","last_change":"2024-09-16T08:45:37.848540+0000","last_active":"2024-09-16T08:46:03.021932+0000","last_peered":"2024-09-16T08:46:03.021932+0000","last_clean":"2024-09-16T08:46:03.021932+0000","last_became_active":"2024-09-16T08:45:37.848203+0000","last_became_peered":"2024-09-16T08:45:37.848203+0000","last_unstale":"2024-09-16T08:46:03.021932+0000","last_undegraded":"2024-09-16T08:46:03.021932+0000","last_fullsized":"2024-09-16T08:46:03.021932+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:34:14.906084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.367758+0000","last_change":"2024-09-16T08:45:39.864881+0000","last_active":"2024-09-16T08:46:03.367758+0000","last_peered":"2024-09-16T08:46:03.367758+0000","last_clean":"2024-09-16T08:46:03.367758+0000","last_became_active":"2024-09-16T08:45:39.864574+0000","last_became_peered":"2024-09-16T08:45:39.864574+0000","last_unstale":"2024-09-16T08:46:03.367758+0000","last_undegraded":"2024-09-16T08:46:03.367758+0000","last_fullsized":"2024-09-16T08:46:03.367758+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:40:28.412183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"62'12","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019539+0000","last_change":"2024-09-16T08:45:35.832462+0000","last_active":"2024-09-16T08:46:03.019539+0000","last_peered":"2024-09-16T08:46:03.019539+0000","last_clean":"2024-09-16T08:46:03.019539+0000","last_became_active":"2024-09-16T08:45:35.832023+0000","last_became_peered":"2024-09-16T08:45:35.832023+0000","last_unstale":"2024-09-16T08:46:03.019539+0000","last_undegraded":"2024-09-16T08:46:03.019539+0000","last_fullsized":"2024-09-16T08:46:03.019539+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:56:11.648976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.368588+0000","last_change":"2024-09-16T08:45:33.815821+0000","last_active":"2024-09-16T08:46:03.368588+0000","last_peered":"2024-09-16T08:46:03.368588+0000","last_clean":"2024-09-16T08:46:03.368588+0000","last_became_active":"2024-09-16T08:45:33.814856+0000","last_became_peered":"2024-09-16T08:45:33.814856+0000","last_unstale":"2024-09-16T08:46:03.368588+0000","last_undegraded":"2024-09-16T08:46:03.368588+0000","last_fullsized":"2024-09-16T08:46:03.368588+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:05.651367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.021458+0000","last_change":"2024-09-16T08:45:37.851977+0000","last_active":"2024-09-16T08:46:03.021458+0000","last_peered":"2024-09-16T08:46:03.021458+0000","last_clean":"2024-09-16T08:46:03.021458+0000","last_became_active":"2024-09-16T08:45:37.851671+0000","last_became_peered":"2024-09-16T08:45:37.851671+0000","last_unstale":"2024-09-16T08:46:03.021458+0000","last_undegraded":"2024-09-16T08:46:03.021458+0000","last_fullsized":"2024-09-16T08:46:03.021458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:24:10.007023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864468+0000","last_change":"2024-09-16T08:45:39.865121+0000","last_active":"2024-09-16T08:46:03.864468+0000","last_peered":"2024-09-16T08:46:03.864468+0000","last_clean":"2024-09-16T08:46:03.864468+0000","last_became_active":"2024-09-16T08:45:39.864805+0000","last_became_peered":"2024-09-16T08:45:39.864805+0000","last_unstale":"2024-09-16T08:46:03.864468+0000","last_undegraded":"2024-09-16T08:46:03.864468+0000","last_fullsized":"2024-09-16T08:46:03.864468+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:55:57.377752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"62'15","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.361977+0000","last_change":"2024-09-16T08:45:35.837361+0000","last_active":"2024-09-16T08:46:03.361977+0000","last_peered":"2024-09-16T08:46:03.361977+0000","last_clean":"2024-09-16T08:46:03.361977+0000","last_became_active":"2024-09-16T08:45:35.837007+0000","last_became_peered":"2024-09-16T08:45:35.837007+0000","last_unstale":"2024-09-16T08:46:03.361977+0000","last_undegraded":"2024-09-16T08:46:03.361977+0000","last_fullsized":"2024-09-16T08:46:03.361977+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:22.135031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"55'2","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.368593+0000","last_change":"2024-09-16T08:45:33.815616+0000","last_active":"2024-09-16T08:46:03.368593+0000","last_peered":"2024-09-16T08:46:03.368593+0000","last_clean":"2024-09-16T08:46:03.368593+0000","last_became_active":"2024-09-16T08:45:33.815224+0000","last_became_peered":"2024-09-16T08:45:33.815224+0000","last_unstale":"2024-09-16T08:46:03.368593+0000","last_undegraded":"2024-09-16T08:46:03.368593+0000","last_fullsized":"2024-09-16T08:46:03.368593+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:51:46.230172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"62'8","reported_seq":40,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.865132+0000","last_change":"2024-09-16T08:45:37.846741+0000","last_active":"2024-09-16T08:46:13.865132+0000","last_peered":"2024-09-16T08:46:13.865132+0000","last_clean":"2024-09-16T08:46:13.865132+0000","last_became_active":"2024-09-16T08:45:37.846299+0000","last_became_peered":"2024-09-16T08:45:37.846299+0000","last_unstale":"2024-09-16T08:46:13.865132+0000","last_undegraded":"2024-09-16T08:46:13.865132+0000","last_fullsized":"2024-09-16T08:46:13.865132+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:59:57.377332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362052+0000","last_change":"2024-09-16T08:45:39.868506+0000","last_active":"2024-09-16T08:46:03.362052+0000","last_peered":"2024-09-16T08:46:03.362052+0000","last_clean":"2024-09-16T08:46:03.362052+0000","last_became_active":"2024-09-16T08:45:39.868238+0000","last_became_peered":"2024-09-16T08:45:39.868238+0000","last_unstale":"2024-09-16T08:46:03.362052+0000","last_undegraded":"2024-09-16T08:46:03.362052+0000","last_fullsized":"2024-09-16T08:46:03.362052+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:46:25.394963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"55'1","reported_seq":36,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.412619+0000","last_change":"2024-09-16T08:45:33.816048+0000","last_active":"2024-09-16T08:46:03.412619+0000","last_peered":"2024-09-16T08:46:03.412619+0000","last_clean":"2024-09-16T08:46:03.412619+0000","last_became_active":"2024-09-16T08:45:33.815745+0000","last_became_peered":"2024-09-16T08:45:33.815745+0000","last_unstale":"2024-09-16T08:46:03.412619+0000","last_undegraded":"2024-09-16T08:46:03.412619+0000","last_fullsized":"2024-09-16T08:46:03.412619+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:09.838323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"62'6","reported_seq":35,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.866010+0000","last_change":"2024-09-16T08:45:35.838918+0000","last_active":"2024-09-16T08:46:03.866010+0000","last_peered":"2024-09-16T08:46:03.866010+0000","last_clean":"2024-09-16T08:46:03.866010+0000","last_became_active":"2024-09-16T08:45:35.838531+0000","last_became_peered":"2024-09-16T08:45:35.838531+0000","last_unstale":"2024-09-16T08:46:03.866010+0000","last_undegraded":"2024-09-16T08:46:03.866010+0000","last_fullsized":"2024-09-16T08:46:03.866010+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:51:02.630687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362568+0000","last_change":"2024-09-16T08:45:37.851852+0000","last_active":"2024-09-16T08:46:03.362568+0000","last_peered":"2024-09-16T08:46:03.362568+0000","last_clean":"2024-09-16T08:46:03.362568+0000","last_became_active":"2024-09-16T08:45:37.851371+0000","last_became_peered":"2024-09-16T08:45:37.851371+0000","last_unstale":"2024-09-16T08:46:03.362568+0000","last_undegraded":"2024-09-16T08:46:03.362568+0000","last_fullsized":"2024-09-16T08:46:03.362568+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:13:03.345415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.367999+0000","last_change":"2024-09-16T08:45:39.871805+0000","last_active":"2024-09-16T08:46:03.367999+0000","last_peered":"2024-09-16T08:46:03.367999+0000","last_clean":"2024-09-16T08:46:03.367999+0000","last_became_active":"2024-09-16T08:45:39.871531+0000","last_became_peered":"2024-09-16T08:45:39.871531+0000","last_unstale":"2024-09-16T08:46:03.367999+0000","last_undegraded":"2024-09-16T08:46:03.367999+0000","last_fullsized":"2024-09-16T08:46:03.367999+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:36:51.283999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020930+0000","last_change":"2024-09-16T08:45:35.840819+0000","last_active":"2024-09-16T08:46:03.020930+0000","last_peered":"2024-09-16T08:46:03.020930+0000","last_clean":"2024-09-16T08:46:03.020930+0000","last_became_active":"2024-09-16T08:45:35.840490+0000","last_became_peered":"2024-09-16T08:45:35.840490+0000","last_unstale":"2024-09-16T08:46:03.020930+0000","last_undegraded":"2024-09-16T08:46:03.020930+0000","last_fullsized":"2024-09-16T08:46:03.020930+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:44.776012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.367952+0000","last_change":"2024-09-16T08:45:33.814341+0000","last_active":"2024-09-16T08:46:03.367952+0000","last_peered":"2024-09-16T08:46:03.367952+0000","last_clean":"2024-09-16T08:46:03.367952+0000","last_became_active":"2024-09-16T08:45:33.813903+0000","last_became_peered":"2024-09-16T08:45:33.813903+0000","last_unstale":"2024-09-16T08:46:03.367952+0000","last_undegraded":"2024-09-16T08:46:03.367952+0000","last_fullsized":"2024-09-16T08:46:03.367952+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:10:41.555349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.865448+0000","last_change":"2024-09-16T08:45:37.856872+0000","last_active":"2024-09-16T08:46:03.865448+0000","last_peered":"2024-09-16T08:46:03.865448+0000","last_clean":"2024-09-16T08:46:03.865448+0000","last_became_active":"2024-09-16T08:45:37.856561+0000","last_became_peered":"2024-09-16T08:45:37.856561+0000","last_unstale":"2024-09-16T08:46:03.865448+0000","last_undegraded":"2024-09-16T08:46:03.865448+0000","last_fullsized":"2024-09-16T08:46:03.865448+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:35.748066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.023730+0000","last_change":"2024-09-16T08:45:39.870005+0000","last_active":"2024-09-16T08:46:03.023730+0000","last_peered":"2024-09-16T08:46:03.023730+0000","last_clean":"2024-09-16T08:46:03.023730+0000","last_became_active":"2024-09-16T08:45:39.869613+0000","last_became_peered":"2024-09-16T08:45:39.869613+0000","last_unstale":"2024-09-16T08:46:03.023730+0000","last_undegraded":"2024-09-16T08:46:03.023730+0000","last_fullsized":"2024-09-16T08:46:03.023730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:43.815457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362565+0000","last_change":"2024-09-16T08:45:35.841050+0000","last_active":"2024-09-16T08:46:03.362565+0000","last_peered":"2024-09-16T08:46:03.362565+0000","last_clean":"2024-09-16T08:46:03.362565+0000","last_became_active":"2024-09-16T08:45:35.840690+0000","last_became_peered":"2024-09-16T08:45:35.840690+0000","last_unstale":"2024-09-16T08:46:03.362565+0000","last_undegraded":"2024-09-16T08:46:03.362565+0000","last_fullsized":"2024-09-16T08:46:03.362565+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:41.045869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.023399+0000","last_change":"2024-09-16T08:45:33.811462+0000","last_active":"2024-09-16T08:46:03.023399+0000","last_peered":"2024-09-16T08:46:03.023399+0000","last_clean":"2024-09-16T08:46:03.023399+0000","last_became_active":"2024-09-16T08:45:33.811171+0000","last_became_peered":"2024-09-16T08:45:33.811171+0000","last_unstale":"2024-09-16T08:46:03.023399+0000","last_undegraded":"2024-09-16T08:46:03.023399+0000","last_fullsized":"2024-09-16T08:46:03.023399+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:36.378011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"62'8","reported_seq":43,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.865186+0000","last_change":"2024-09-16T08:45:37.851783+0000","last_active":"2024-09-16T08:46:13.865186+0000","last_peered":"2024-09-16T08:46:13.865186+0000","last_clean":"2024-09-16T08:46:13.865186+0000","last_became_active":"2024-09-16T08:45:37.851444+0000","last_became_peered":"2024-09-16T08:45:37.851444+0000","last_unstale":"2024-09-16T08:46:13.865186+0000","last_undegraded":"2024-09-16T08:46:13.865186+0000","last_fullsized":"2024-09-16T08:46:13.865186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:45:15.301516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020597+0000","last_change":"2024-09-16T08:45:39.870668+0000","last_active":"2024-09-16T08:46:03.020597+0000","last_peered":"2024-09-16T08:46:03.020597+0000","last_clean":"2024-09-16T08:46:03.020597+0000","last_became_active":"2024-09-16T08:45:39.870415+0000","last_became_peered":"2024-09-16T08:45:39.870415+0000","last_unstale":"2024-09-16T08:46:03.020597+0000","last_undegraded":"2024-09-16T08:46:03.020597+0000","last_fullsized":"2024-09-16T08:46:03.020597+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:55.494362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"62'10","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864813+0000","last_change":"2024-09-16T08:45:35.839930+0000","last_active":"2024-09-16T08:46:03.864813+0000","last_peered":"2024-09-16T08:46:03.864813+0000","last_clean":"2024-09-16T08:46:03.864813+0000","last_became_active":"2024-09-16T08:45:35.839635+0000","last_became_peered":"2024-09-16T08:45:35.839635+0000","last_unstale":"2024-09-16T08:46:03.864813+0000","last_undegraded":"2024-09-16T08:46:03.864813+0000","last_fullsized":"2024-09-16T08:46:03.864813+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:27:34.133426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.368217+0000","last_change":"2024-09-16T08:45:33.814268+0000","last_active":"2024-09-16T08:46:03.368217+0000","last_peered":"2024-09-16T08:46:03.368217+0000","last_clean":"2024-09-16T08:46:03.368217+0000","last_became_active":"2024-09-16T08:45:33.813539+0000","last_became_peered":"2024-09-16T08:45:33.813539+0000","last_unstale":"2024-09-16T08:46:03.368217+0000","last_undegraded":"2024-09-16T08:46:03.368217+0000","last_fullsized":"2024-09-16T08:46:03.368217+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:56.773850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"62'8","reported_seq":40,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:13.865247+0000","last_change":"2024-09-16T08:45:37.849710+0000","last_active":"2024-09-16T08:46:13.865247+0000","last_peered":"2024-09-16T08:46:13.865247+0000","last_clean":"2024-09-16T08:46:13.865247+0000","last_became_active":"2024-09-16T08:45:37.849378+0000","last_became_peered":"2024-09-16T08:45:37.849378+0000","last_unstale":"2024-09-16T08:46:13.865247+0000","last_undegraded":"2024-09-16T08:46:13.865247+0000","last_fullsized":"2024-09-16T08:46:13.865247+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:22:57.496083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020022+0000","last_change":"2024-09-16T08:45:39.863788+0000","last_active":"2024-09-16T08:46:03.020022+0000","last_peered":"2024-09-16T08:46:03.020022+0000","last_clean":"2024-09-16T08:46:03.020022+0000","last_became_active":"2024-09-16T08:45:39.863428+0000","last_became_peered":"2024-09-16T08:45:39.863428+0000","last_unstale":"2024-09-16T08:46:03.020022+0000","last_undegraded":"2024-09-16T08:46:03.020022+0000","last_fullsized":"2024-09-16T08:46:03.020022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:29.146892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"62'11","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020377+0000","last_change":"2024-09-16T08:45:35.833430+0000","last_active":"2024-09-16T08:46:03.020377+0000","last_peered":"2024-09-16T08:46:03.020377+0000","last_clean":"2024-09-16T08:46:03.020377+0000","last_became_active":"2024-09-16T08:45:35.833112+0000","last_became_peered":"2024-09-16T08:45:35.833112+0000","last_unstale":"2024-09-16T08:46:03.020377+0000","last_undegraded":"2024-09-16T08:46:03.020377+0000","last_fullsized":"2024-09-16T08:46:03.020377+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:36:41.826634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020471+0000","last_change":"2024-09-16T08:45:33.817133+0000","last_active":"2024-09-16T08:46:03.020471+0000","last_peered":"2024-09-16T08:46:03.020471+0000","last_clean":"2024-09-16T08:46:03.020471+0000","last_became_active":"2024-09-16T08:45:33.816860+0000","last_became_peered":"2024-09-16T08:45:33.816860+0000","last_unstale":"2024-09-16T08:46:03.020471+0000","last_undegraded":"2024-09-16T08:46:03.020471+0000","last_fullsized":"2024-09-16T08:46:03.020471+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:26.123965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.345632+0000","last_change":"2024-09-16T08:45:37.855458+0000","last_active":"2024-09-16T08:46:03.345632+0000","last_peered":"2024-09-16T08:46:03.345632+0000","last_clean":"2024-09-16T08:46:03.345632+0000","last_became_active":"2024-09-16T08:45:37.854292+0000","last_became_peered":"2024-09-16T08:45:37.854292+0000","last_unstale":"2024-09-16T08:46:03.345632+0000","last_undegraded":"2024-09-16T08:46:03.345632+0000","last_fullsized":"2024-09-16T08:46:03.345632+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:01:10.548237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864676+0000","last_change":"2024-09-16T08:45:39.869637+0000","last_active":"2024-09-16T08:46:03.864676+0000","last_peered":"2024-09-16T08:46:03.864676+0000","last_clean":"2024-09-16T08:46:03.864676+0000","last_became_active":"2024-09-16T08:45:39.869352+0000","last_became_peered":"2024-09-16T08:45:39.869352+0000","last_unstale":"2024-09-16T08:46:03.864676+0000","last_undegraded":"2024-09-16T08:46:03.864676+0000","last_fullsized":"2024-09-16T08:46:03.864676+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:44:46.751050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.345893+0000","last_change":"2024-09-16T08:45:35.833229+0000","last_active":"2024-09-16T08:46:03.345893+0000","last_peered":"2024-09-16T08:46:03.345893+0000","last_clean":"2024-09-16T08:46:03.345893+0000","last_became_active":"2024-09-16T08:45:35.832945+0000","last_became_peered":"2024-09-16T08:45:35.832945+0000","last_unstale":"2024-09-16T08:46:03.345893+0000","last_undegraded":"2024-09-16T08:46:03.345893+0000","last_fullsized":"2024-09-16T08:46:03.345893+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:08.228692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.368350+0000","last_change":"2024-09-16T08:45:33.815884+0000","last_active":"2024-09-16T08:46:03.368350+0000","last_peered":"2024-09-16T08:46:03.368350+0000","last_clean":"2024-09-16T08:46:03.368350+0000","last_became_active":"2024-09-16T08:45:33.815590+0000","last_became_peered":"2024-09-16T08:45:33.815590+0000","last_unstale":"2024-09-16T08:46:03.368350+0000","last_undegraded":"2024-09-16T08:46:03.368350+0000","last_fullsized":"2024-09-16T08:46:03.368350+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:04.048331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864577+0000","last_change":"2024-09-16T08:45:37.854982+0000","last_active":"2024-09-16T08:46:03.864577+0000","last_peered":"2024-09-16T08:46:03.864577+0000","last_clean":"2024-09-16T08:46:03.864577+0000","last_became_active":"2024-09-16T08:45:37.854701+0000","last_became_peered":"2024-09-16T08:45:37.854701+0000","last_unstale":"2024-09-16T08:46:03.864577+0000","last_undegraded":"2024-09-16T08:46:03.864577+0000","last_fullsized":"2024-09-16T08:46:03.864577+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:05.952109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.019184+0000","last_change":"2024-09-16T08:45:39.866984+0000","last_active":"2024-09-16T08:46:03.019184+0000","last_peered":"2024-09-16T08:46:03.019184+0000","last_clean":"2024-09-16T08:46:03.019184+0000","last_became_active":"2024-09-16T08:45:39.866635+0000","last_became_peered":"2024-09-16T08:45:39.866635+0000","last_unstale":"2024-09-16T08:46:03.019184+0000","last_undegraded":"2024-09-16T08:46:03.019184+0000","last_fullsized":"2024-09-16T08:46:03.019184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:01:03.825861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"62'11","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864364+0000","last_change":"2024-09-16T08:45:35.837782+0000","last_active":"2024-09-16T08:46:03.864364+0000","last_peered":"2024-09-16T08:46:03.864364+0000","last_clean":"2024-09-16T08:46:03.864364+0000","last_became_active":"2024-09-16T08:45:35.837399+0000","last_became_peered":"2024-09-16T08:45:35.837399+0000","last_unstale":"2024-09-16T08:46:03.864364+0000","last_undegraded":"2024-09-16T08:46:03.864364+0000","last_fullsized":"2024-09-16T08:46:03.864364+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:48:40.456091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.362648+0000","last_change":"2024-09-16T08:45:33.818022+0000","last_active":"2024-09-16T08:46:03.362648+0000","last_peered":"2024-09-16T08:46:03.362648+0000","last_clean":"2024-09-16T08:46:03.362648+0000","last_became_active":"2024-09-16T08:45:33.817721+0000","last_became_peered":"2024-09-16T08:45:33.817721+0000","last_unstale":"2024-09-16T08:46:03.362648+0000","last_undegraded":"2024-09-16T08:46:03.362648+0000","last_fullsized":"2024-09-16T08:46:03.362648+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:51:58.540936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.367569+0000","last_change":"2024-09-16T08:45:37.846757+0000","last_active":"2024-09-16T08:46:03.367569+0000","last_peered":"2024-09-16T08:46:03.367569+0000","last_clean":"2024-09-16T08:46:03.367569+0000","last_became_active":"2024-09-16T08:45:37.846293+0000","last_became_peered":"2024-09-16T08:45:37.846293+0000","last_unstale":"2024-09-16T08:46:03.367569+0000","last_undegraded":"2024-09-16T08:46:03.367569+0000","last_fullsized":"2024-09-16T08:46:03.367569+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:56.864171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864728+0000","last_change":"2024-09-16T08:45:39.868192+0000","last_active":"2024-09-16T08:46:03.864728+0000","last_peered":"2024-09-16T08:46:03.864728+0000","last_clean":"2024-09-16T08:46:03.864728+0000","last_became_active":"2024-09-16T08:45:39.867893+0000","last_became_peered":"2024-09-16T08:45:39.867893+0000","last_unstale":"2024-09-16T08:46:03.864728+0000","last_undegraded":"2024-09-16T08:46:03.864728+0000","last_fullsized":"2024-09-16T08:46:03.864728+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:07.059601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"62'4","reported_seq":32,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864185+0000","last_change":"2024-09-16T08:45:35.835301+0000","last_active":"2024-09-16T08:46:03.864185+0000","last_peered":"2024-09-16T08:46:03.864185+0000","last_clean":"2024-09-16T08:46:03.864185+0000","last_became_active":"2024-09-16T08:45:35.834963+0000","last_became_peered":"2024-09-16T08:45:35.834963+0000","last_unstale":"2024-09-16T08:46:03.864185+0000","last_undegraded":"2024-09-16T08:46:03.864185+0000","last_fullsized":"2024-09-16T08:46:03.864185+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:29.824770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.024570+0000","last_change":"2024-09-16T08:45:33.809757+0000","last_active":"2024-09-16T08:46:03.024570+0000","last_peered":"2024-09-16T08:46:03.024570+0000","last_clean":"2024-09-16T08:46:03.024570+0000","last_became_active":"2024-09-16T08:45:33.809444+0000","last_became_peered":"2024-09-16T08:45:33.809444+0000","last_unstale":"2024-09-16T08:46:03.024570+0000","last_undegraded":"2024-09-16T08:46:03.024570+0000","last_fullsized":"2024-09-16T08:46:03.024570+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:10.444170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.412547+0000","last_change":"2024-09-16T08:45:37.849382+0000","last_active":"2024-09-16T08:46:03.412547+0000","last_peered":"2024-09-16T08:46:03.412547+0000","last_clean":"2024-09-16T08:46:03.412547+0000","last_became_active":"2024-09-16T08:45:37.849095+0000","last_became_peered":"2024-09-16T08:45:37.849095+0000","last_unstale":"2024-09-16T08:46:03.412547+0000","last_undegraded":"2024-09-16T08:46:03.412547+0000","last_fullsized":"2024-09-16T08:46:03.412547+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:14:49.842211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.367698+0000","last_change":"2024-09-16T08:45:39.872542+0000","last_active":"2024-09-16T08:46:03.367698+0000","last_peered":"2024-09-16T08:46:03.367698+0000","last_clean":"2024-09-16T08:46:03.367698+0000","last_became_active":"2024-09-16T08:45:39.872265+0000","last_became_peered":"2024-09-16T08:45:39.872265+0000","last_unstale":"2024-09-16T08:46:03.367698+0000","last_undegraded":"2024-09-16T08:46:03.367698+0000","last_fullsized":"2024-09-16T08:46:03.367698+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:38.307316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.346269+0000","last_change":"2024-09-16T08:45:39.871358+0000","last_active":"2024-09-16T08:46:03.346269+0000","last_peered":"2024-09-16T08:46:03.346269+0000","last_clean":"2024-09-16T08:46:03.346269+0000","last_became_active":"2024-09-16T08:45:39.871018+0000","last_became_peered":"2024-09-16T08:45:39.871018+0000","last_unstale":"2024-09-16T08:46:03.346269+0000","last_undegraded":"2024-09-16T08:46:03.346269+0000","last_fullsized":"2024-09-16T08:46:03.346269+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:38.834869+0000","last_clean_scrub_stamp":"2024-09-16T08:45:38.834869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:09:14.465197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.864373+0000","last_change":"2024-09-16T08:45:33.805599+0000","last_active":"2024-09-16T08:46:03.864373+0000","last_peered":"2024-09-16T08:46:03.864373+0000","last_clean":"2024-09-16T08:46:03.864373+0000","last_became_active":"2024-09-16T08:45:33.805262+0000","last_became_peered":"2024-09-16T08:45:33.805262+0000","last_unstale":"2024-09-16T08:46:03.864373+0000","last_undegraded":"2024-09-16T08:46:03.864373+0000","last_fullsized":"2024-09-16T08:46:03.864373+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:32.789278+0000","last_clean_scrub_stamp":"2024-09-16T08:45:32.789278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:15:24.051369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"62'11","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.413104+0000","last_change":"2024-09-16T08:45:35.833205+0000","last_active":"2024-09-16T08:46:03.413104+0000","last_peered":"2024-09-16T08:46:03.413104+0000","last_clean":"2024-09-16T08:46:03.413104+0000","last_became_active":"2024-09-16T08:45:35.832892+0000","last_became_peered":"2024-09-16T08:45:35.832892+0000","last_unstale":"2024-09-16T08:46:03.413104+0000","last_undegraded":"2024-09-16T08:46:03.413104+0000","last_fullsized":"2024-09-16T08:46:03.413104+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:34.805688+0000","last_clean_scrub_stamp":"2024-09-16T08:45:34.805688+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:07:29.314959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T08:46:03.020132+0000","last_change":"2024-09-16T08:45:37.846900+0000","last_active":"2024-09-16T08:46:03.020132+0000","last_peered":"2024-09-16T08:46:03.020132+0000","last_clean":"2024-09-16T08:46:03.020132+0000","last_became_active":"2024-09-16T08:45:37.846616+0000","last_became_peered":"2024-09-16T08:45:37.846616+0000","last_unstale":"2024-09-16T08:46:03.020132+0000","last_undegraded":"2024-09-16T08:46:03.020132+0000","last_fullsized":"2024-09-16T08:46:03.020132+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:45:36.823872+0000","last_clean_scrub_stamp":"2024-09-16T08:45:36.823872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:47.867331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":35,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":50,"seq":214748364812,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32396,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712756,"statfs":{"total":95995035648,"available":95961862144,"internally_reserved":0,"allocated":1241088,"data_stored":833976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561041,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1220608,"data_stored":833314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757271,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717508,"statfs":{"total":95995035648,"available":95966728192,"internally_reserved":0,"allocated":630784,"data_stored":243305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953503,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31832,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713320,"statfs":{"total":95995035648,"available":95962439680,"internally_reserved":0,"allocated":659456,"data_stored":244121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149732,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27680,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717472,"statfs":{"total":95995035648,"available":95966691328,"internally_reserved":0,"allocated":667648,"data_stored":244407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411371,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27632,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717520,"statfs":{"total":95995035648,"available":95966740480,"internally_reserved":0,"allocated":618496,"data_stored":242658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31860,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713292,"statfs":{"total":95995035648,"available":95962411008,"internally_reserved":0,"allocated":688128,"data_stored":244295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32404,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712748,"statfs":{"total":95995035648,"available":95961853952,"internally_reserved":0,"allocated":1249280,"data_stored":835101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T08:46:22.896 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T08:46:22.897 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T08:46:22.897 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-16T08:46:22.897 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-16T08:46:23.310 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:46:24.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.093+0000 7f391f798640 1 -- 172.21.15.160:0/2265069659 >> v1:172.21.15.160:6789/0 conn(0x7f391810b6b0 legacy=0x7f391810daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:24.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.094+0000 7f391f798640 1 -- 172.21.15.160:0/2265069659 shutdown_connections 2024-09-16T08:46:24.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.094+0000 7f391f798640 1 -- 172.21.15.160:0/2265069659 >> 172.21.15.160:0/2265069659 conn(0x7f39180fe590 msgr2=0x7f39181009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:24.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.094+0000 7f391f798640 1 -- 172.21.15.160:0/2265069659 shutdown_connections 2024-09-16T08:46:24.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.094+0000 7f391f798640 1 -- 172.21.15.160:0/2265069659 wait complete. 2024-09-16T08:46:24.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.094+0000 7f391f798640 1 Processor -- start 2024-09-16T08:46:24.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.094+0000 7f391f798640 1 -- start start 2024-09-16T08:46:24.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f391f798640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f39181a7730 con 0x7f3918102b40 2024-09-16T08:46:24.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f391f798640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f39181a8930 con 0x7f391810f250 2024-09-16T08:46:24.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f391f798640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f39181a9b30 con 0x7f391810b6b0 2024-09-16T08:46:24.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f391d534640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f3918102b40 0x7f391810e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:45958/0 (socket says 172.21.15.160:45958) 2024-09-16T08:46:24.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f391cd33640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f391810b6b0 0x7f39181a26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:43572/0 (socket says 172.21.15.160:43572) 2024-09-16T08:46:24.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f391d534640 1 -- 172.21.15.160:0/3273762083 learned_addr learned my addr 172.21.15.160:0/3273762083 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T08:46:24.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2799864347 0 0) 0x7f39181a9b30 con 0x7f391810b6b0 2024-09-16T08:46:24.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3904003520 con 0x7f391810b6b0 2024-09-16T08:46:24.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 476272401 0 0) 0x7f39181a7730 con 0x7f3918102b40 2024-09-16T08:46:24.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39181a9b30 con 0x7f3918102b40 2024-09-16T08:46:24.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4026573171 0 0) 0x7f39181a8930 con 0x7f391810f250 2024-09-16T08:46:24.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39181a7730 con 0x7f391810f250 2024-09-16T08:46:24.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2420387714 0 0) 0x7f3904003520 con 0x7f391810b6b0 2024-09-16T08:46:24.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f39181a8930 con 0x7f391810b6b0 2024-09-16T08:46:24.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f3914001650 con 0x7f391810b6b0 2024-09-16T08:46:24.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2407351718 0 0) 0x7f39181a9b30 con 0x7f3918102b40 2024-09-16T08:46:24.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3904003520 con 0x7f3918102b40 2024-09-16T08:46:24.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f3908001310 con 0x7f3918102b40 2024-09-16T08:46:24.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3808652394 0 0) 0x7f39181a7730 con 0x7f391810f250 2024-09-16T08:46:24.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f39181a9b30 con 0x7f391810f250 2024-09-16T08:46:24.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f3910001b70 con 0x7f391810f250 2024-09-16T08:46:24.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 30542933 0 0) 0x7f39181a8930 con 0x7f391810b6b0 2024-09-16T08:46:24.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 >> v1:172.21.15.169:6789/0 conn(0x7f391810f250 legacy=0x7f39181a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:24.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 >> v1:172.21.15.160:6789/0 conn(0x7f3918102b40 legacy=0x7f391810e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:24.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.095+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39181aadd0 con 0x7f391810b6b0 2024-09-16T08:46:24.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.096+0000 7f391f798640 1 -- 172.21.15.160:0/3273762083 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f39181a7960 con 0x7f391810b6b0 2024-09-16T08:46:24.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.096+0000 7f391f798640 1 -- 172.21.15.160:0/3273762083 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f39181a7f90 con 0x7f391810b6b0 2024-09-16T08:46:24.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.096+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f3914001150 con 0x7f391810b6b0 2024-09-16T08:46:24.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.096+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f3914004b70 con 0x7f391810b6b0 2024-09-16T08:46:24.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.097+0000 7f391f798640 1 -- 172.21.15.160:0/3273762083 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38fc0050b0 con 0x7f391810b6b0 2024-09-16T08:46:24.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.097+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7f39140035f0 con 0x7f391810b6b0 2024-09-16T08:46:24.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.098+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(64..64 src has 1..64) ==== 6172+0+0 (unknown 440296652 0 0) 0x7f3914099160 con 0x7f391810b6b0 2024-09-16T08:46:24.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.102+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f3914064770 con 0x7f391810b6b0 2024-09-16T08:46:24.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.320+0000 7f391f798640 1 -- 172.21.15.160:0/3273762083 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f38fc0053a0 con 0x7f391810b6b0 2024-09-16T08:46:24.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.320+0000 7f390e7fc640 1 -- 172.21.15.160:0/3273762083 <== mon.2 v1:172.21.15.160:6790/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (unknown 3730786198 0 4185958460) 0x7f39140696b0 con 0x7f391810b6b0 2024-09-16T08:46:24.322 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-16T08:46:24.322 INFO:teuthology.orchestra.run.smithi160.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T08:46:24.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.323+0000 7f391f798640 1 -- 172.21.15.160:0/3273762083 >> v1:172.21.15.160:6800/2859204856 conn(0x7f390407bbf0 legacy=0x7f390407e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:24.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.323+0000 7f391f798640 1 -- 172.21.15.160:0/3273762083 >> v1:172.21.15.160:6790/0 conn(0x7f391810b6b0 legacy=0x7f39181a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:46:24.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.323+0000 7f391f798640 1 -- 172.21.15.160:0/3273762083 shutdown_connections 2024-09-16T08:46:24.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.323+0000 7f391f798640 1 -- 172.21.15.160:0/3273762083 >> 172.21.15.160:0/3273762083 conn(0x7f39180fe590 msgr2=0x7f3918106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:46:24.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.323+0000 7f391f798640 1 -- 172.21.15.160:0/3273762083 shutdown_connections 2024-09-16T08:46:24.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-16T08:46:24.323+0000 7f391f798640 1 -- 172.21.15.160:0/3273762083 wait complete. 2024-09-16T08:46:24.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:24 smithi169 ceph-mon[33465]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:24.663 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:24 smithi160 ceph-mon[30137]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:24.663 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:24 smithi160 ceph-mon[40028]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:25.099 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:46:24 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:46:25.316 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:46:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:46:25.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3273762083' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-16T08:46:25.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3273762083' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-16T08:46:25.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3273762083' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-16T08:46:25.753 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-16T08:46:25.753 INFO:tasks.cephadm:Setup complete, yielding 2024-09-16T08:46:25.753 INFO:teuthology.run_tasks:Running task workunit... 2024-09-16T08:46:25.770 INFO:tasks.workunit:Pulling workunits from ref 8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:46:25.771 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-16T08:46:25.771 DEBUG:teuthology.orchestra.run.smithi160:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-16T08:46:25.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:46:25.789 INFO:teuthology.orchestra.run.smithi160.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-16T08:46:25.789 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-16T08:46:25.847 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-16T08:46:25.847 DEBUG:teuthology.orchestra.run.smithi160:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-16T08:46:25.903 INFO:tasks.workunit:timeout=3h 2024-09-16T08:46:25.903 INFO:tasks.workunit:cleanup=True 2024-09-16T08:46:25.903 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:46:25.960 INFO:tasks.workunit.client.0.smithi160.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-16T08:46:26.556 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:26.556 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:26 smithi160 ceph-mon[30137]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:26.557 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:26.557 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:26 smithi160 ceph-mon[40028]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:26.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:26.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:26 smithi169 ceph-mon[33465]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:28.321 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:28 smithi160 systemd[1]: Starting Ceph alertmanager.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T08:46:28.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:28 smithi169 ceph-mon[33465]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:28.675 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:28 smithi160 ceph-mon[30137]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:28.676 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:28 smithi160 ceph-mon[40028]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:28.676 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:28 smithi160 podman[73882]: 2024-09-16 08:46:28.289206492 +0000 UTC m=+0.016467904 image pull 11f11916f8cdfb17c1da74b20deb2075d778357b4d60474d82453303f8d461f5 quay.io/prometheus/alertmanager:v0.27.0 2024-09-16T08:46:28.676 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:28 smithi160 podman[73882]: 2024-09-16 08:46:28.398890461 +0000 UTC m=+0.126151873 volume create e21079aa62dd775e0215bace7beb4bb8900e7530966bccbce6df24f342e8fddf 2024-09-16T08:46:28.677 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:28 smithi160 podman[73882]: 2024-09-16 08:46:28.499290348 +0000 UTC m=+0.226551759 container create acebf1ef0331f6201baeb71373f5cc3de1aa748b5649bf4b6a35bfe9d5884411 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:46:29.198 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:28 smithi160 podman[73882]: 2024-09-16 08:46:28.768351985 +0000 UTC m=+0.495613397 container init acebf1ef0331f6201baeb71373f5cc3de1aa748b5649bf4b6a35bfe9d5884411 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:46:29.198 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:28 smithi160 podman[73882]: 2024-09-16 08:46:28.773175536 +0000 UTC m=+0.500436949 container start acebf1ef0331f6201baeb71373f5cc3de1aa748b5649bf4b6a35bfe9d5884411 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:46:29.198 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:28 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[73896]: ts=2024-09-16T08:46:28.784Z caller=main.go:181 level=info msg="Starting Alertmanager" version="(version=0.27.0, branch=HEAD, revision=0aa3c2aad14cff039931923ab16b26b7481783b5)" 2024-09-16T08:46:29.198 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:28 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[73896]: ts=2024-09-16T08:46:28.784Z caller=main.go:182 level=info build_context="(go=go1.21.7, platform=linux/amd64, user=root@22cd11f671e9, date=20240228-11:51:20, tags=netgo)" 2024-09-16T08:46:29.198 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:28 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[73896]: ts=2024-09-16T08:46:28.785Z caller=cluster.go:186 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.160 port=9094 2024-09-16T08:46:29.198 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:28 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[73896]: ts=2024-09-16T08:46:28.786Z caller=cluster.go:683 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-16T08:46:29.199 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:28 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[73896]: ts=2024-09-16T08:46:28.807Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T08:46:29.199 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:28 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[73896]: ts=2024-09-16T08:46:28.808Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T08:46:29.199 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:28 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[73896]: ts=2024-09-16T08:46:28.809Z caller=tls_config.go:313 level=info msg="Listening on" address=[::]:9093 2024-09-16T08:46:29.199 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:28 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[73896]: ts=2024-09-16T08:46:28.809Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-16T08:46:29.199 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:28 smithi160 bash[73882]: acebf1ef0331f6201baeb71373f5cc3de1aa748b5649bf4b6a35bfe9d5884411 2024-09-16T08:46:29.199 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:28 smithi160 systemd[1]: Started Ceph alertmanager.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:46:30.184 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:29 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:30.184 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:29 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:30.184 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:29 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:30.184 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:29 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:30.185 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:29 smithi160 ceph-mon[30137]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T08:46:30.185 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:29 smithi160 ceph-mon[30137]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:30.185 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:29 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:30.185 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:29 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:30.185 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:29 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:30.185 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:29 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:30.185 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:29 smithi160 ceph-mon[40028]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T08:46:30.185 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:29 smithi160 ceph-mon[40028]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:30.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:29 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:30.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:29 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:30.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:29 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:30.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:29 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:30.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:29 smithi169 ceph-mon[33465]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T08:46:30.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:29 smithi169 ceph-mon[33465]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:30.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:30 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:46:30] ENGINE Bus STOPPING 2024-09-16T08:46:30.786 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:30 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:46:30] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T08:46:30.786 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:30 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:46:30] ENGINE Bus STOPPED 2024-09-16T08:46:30.786 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:30 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:46:30] ENGINE Bus STARTING 2024-09-16T08:46:30.786 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:30 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:46:30] ENGINE Serving on http://:::9283 2024-09-16T08:46:30.786 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:30 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:46:30] ENGINE Bus STARTED 2024-09-16T08:46:31.166 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:30 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[73896]: ts=2024-09-16T08:46:30.786Z caller=cluster.go:708 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000153013s 2024-09-16T08:46:31.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:31 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:31.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:31 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:31.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:31 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-16T08:46:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:31 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T08:46:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:31 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:31 smithi160 ceph-mon[30137]: Deploying daemon grafana.a on smithi169 2024-09-16T08:46:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:31 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:46:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:31 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:46:31.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:31 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:31.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:31 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:31.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:31 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-16T08:46:31.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:31 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T08:46:31.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:31 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:31.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:31 smithi160 ceph-mon[40028]: Deploying daemon grafana.a on smithi169 2024-09-16T08:46:31.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:31 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:46:31.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:31 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:46:31.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:31 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:31.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:31 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:31.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:31 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-16T08:46:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:31 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T08:46:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:31 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:31 smithi169 ceph-mon[33465]: Deploying daemon grafana.a on smithi169 2024-09-16T08:46:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:31 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:46:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:31 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:46:32.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:32 smithi160 ceph-mon[30137]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:32.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:32 smithi160 ceph-mon[40028]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:32.500 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:32 smithi169 ceph-mon[33465]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:34.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:34 smithi169 ceph-mon[33465]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:34.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:46:34.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:34 smithi160 ceph-mon[30137]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:46:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:34 smithi160 ceph-mon[40028]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:46:35.099 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:46:34 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:46:35.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:46:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:46:36.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:36 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:36.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:36 smithi169 ceph-mon[33465]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:36.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:36 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:36.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:36 smithi160 ceph-mon[30137]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:36 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:36 smithi160 ceph-mon[40028]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:38.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:38 smithi169 ceph-mon[33465]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:38.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:38 smithi160 ceph-mon[30137]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:38 smithi160 ceph-mon[40028]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:39.197 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:38 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[73896]: ts=2024-09-16T08:46:38.787Z caller=cluster.go:700 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.001080829s 2024-09-16T08:46:40.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:40 smithi169 ceph-mon[33465]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:40.600 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:40 smithi169 systemd[1]: Starting Ceph grafana.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T08:46:40.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:40 smithi160 ceph-mon[30137]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:40 smithi160 ceph-mon[40028]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:41.027 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:40 smithi169 podman[63471]: 2024-09-16 08:46:40.597555381 +0000 UTC m=+0.016840597 image pull c8b91775d855b99270fc5d22f3c6737e8cca01ef4c25c8b0362295e0746fa39b quay.io/ceph/grafana:10.4.0 2024-09-16T08:46:41.027 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:40 smithi169 podman[63471]: 2024-09-16 08:46:40.833534599 +0000 UTC m=+0.252819844 container create 9af72bc33191d0a6d739b3ce68288a88c0a27350667b293f99a67b2429cd05bc (image=quay.io/ceph/grafana:10.4.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-16T08:46:41.027 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 podman[63471]: 2024-09-16 08:46:41.027223677 +0000 UTC m=+0.446508881 container init 9af72bc33191d0a6d739b3ce68288a88c0a27350667b293f99a67b2429cd05bc (image=quay.io/ceph/grafana:10.4.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-16T08:46:41.278 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 podman[63471]: 2024-09-16 08:46:41.032233345 +0000 UTC m=+0.451518559 container start 9af72bc33191d0a6d739b3ce68288a88c0a27350667b293f99a67b2429cd05bc (image=quay.io/ceph/grafana:10.4.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-16T08:46:41.278 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 bash[63471]: 9af72bc33191d0a6d739b3ce68288a88c0a27350667b293f99a67b2429cd05bc 2024-09-16T08:46:41.278 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 systemd[1]: Started Ceph grafana.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:46:41.279 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.159576354Z level=info msg="Starting Grafana" version=10.4.0 commit=03f502a94d17f7dc4e6c34acdf8428aedd986e4c branch=HEAD compiled=2024-09-16T08:46:41Z 2024-09-16T08:46:41.279 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.15979543Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-16T08:46:41.279 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.15980286Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-16T08:46:41.279 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.159806349Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" 2024-09-16T08:46:41.279 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.159809096Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" 2024-09-16T08:46:41.279 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.15981178Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" 2024-09-16T08:46:41.280 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.159814295Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" 2024-09-16T08:46:41.280 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.15981695Z level=info msg="Config overridden from command line" arg="default.log.mode=console" 2024-09-16T08:46:41.280 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.159819684Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-16T08:46:41.280 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.159822201Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-16T08:46:41.280 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.15982523Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-16T08:46:41.281 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.159827752Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-16T08:46:41.281 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.159830969Z level=info msg=Target target=[all] 2024-09-16T08:46:41.281 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.15983617Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-16T08:46:41.281 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.159838625Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-16T08:46:41.281 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.159841051Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-16T08:46:41.285 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.159843994Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-16T08:46:41.285 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.159846441Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-16T08:46:41.286 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=settings t=2024-09-16T08:46:41.159852223Z level=info msg="App mode production" 2024-09-16T08:46:41.286 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=sqlstore t=2024-09-16T08:46:41.160094296Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-16T08:46:41.286 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=sqlstore t=2024-09-16T08:46:41.160107075Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-16T08:46:41.286 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.160603526Z level=info msg="Starting DB migrations" 2024-09-16T08:46:41.287 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.161479022Z level=info msg="Executing migration" id="create migration_log table" 2024-09-16T08:46:41.287 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.162220515Z level=info msg="Migration successfully executed" id="create migration_log table" duration=741.393µs 2024-09-16T08:46:41.287 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.16269171Z level=info msg="Executing migration" id="create user table" 2024-09-16T08:46:41.287 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.163227679Z level=info msg="Migration successfully executed" id="create user table" duration=536.171µs 2024-09-16T08:46:41.287 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.163647074Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-16T08:46:41.288 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.1641719Z level=info msg="Migration successfully executed" id="add unique index user.login" duration=523.733µs 2024-09-16T08:46:41.288 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.164579003Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-16T08:46:41.288 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.165060757Z level=info msg="Migration successfully executed" id="add unique index user.email" duration=481.746µs 2024-09-16T08:46:41.288 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.165451525Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-16T08:46:41.288 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.16594844Z level=info msg="Migration successfully executed" id="drop index UQE_user_login - v1" duration=496.898µs 2024-09-16T08:46:41.289 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.166401556Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-16T08:46:41.289 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.166895189Z level=info msg="Migration successfully executed" id="drop index UQE_user_email - v1" duration=494.799µs 2024-09-16T08:46:41.289 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.167333722Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-16T08:46:41.289 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.168951028Z level=info msg="Migration successfully executed" id="Rename table user to user_v1 - v1" duration=1.616823ms 2024-09-16T08:46:41.289 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.16941367Z level=info msg="Executing migration" id="create user table v2" 2024-09-16T08:46:41.289 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.169996536Z level=info msg="Migration successfully executed" id="create user table v2" duration=582.679µs 2024-09-16T08:46:41.290 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.170361711Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-16T08:46:41.290 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.17083184Z level=info msg="Migration successfully executed" id="create index UQE_user_login - v2" duration=471.12µs 2024-09-16T08:46:41.290 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.171209121Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-16T08:46:41.290 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.171660813Z level=info msg="Migration successfully executed" id="create index UQE_user_email - v2" duration=451.615µs 2024-09-16T08:46:41.290 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.17209943Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-16T08:46:41.291 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.172384598Z level=info msg="Migration successfully executed" id="copy data_source v1 to v2" duration=285.331µs 2024-09-16T08:46:41.291 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.172722097Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-16T08:46:41.291 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.173122954Z level=info msg="Migration successfully executed" id="Drop old table user_v1" duration=400.649µs 2024-09-16T08:46:41.291 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.173493402Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-16T08:46:41.291 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.17424996Z level=info msg="Migration successfully executed" id="Add column help_flags1 to user table" duration=756.393µs 2024-09-16T08:46:41.292 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.174614091Z level=info msg="Executing migration" id="Update user table charset" 2024-09-16T08:46:41.292 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.174631929Z level=info msg="Migration successfully executed" id="Update user table charset" duration=19.106µs 2024-09-16T08:46:41.292 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.175054758Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-16T08:46:41.292 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.175784605Z level=info msg="Migration successfully executed" id="Add last_seen_at column to user" duration=729.465µs 2024-09-16T08:46:41.292 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.176148893Z level=info msg="Executing migration" id="Add missing user data" 2024-09-16T08:46:41.293 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.176310279Z level=info msg="Migration successfully executed" id="Add missing user data" duration=161.565µs 2024-09-16T08:46:41.293 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.176721004Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-16T08:46:41.293 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.177450534Z level=info msg="Migration successfully executed" id="Add is_disabled column to user" duration=729.167µs 2024-09-16T08:46:41.293 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.177816926Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-16T08:46:41.293 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.178294064Z level=info msg="Migration successfully executed" id="Add index user.login/user.email" duration=477.017µs 2024-09-16T08:46:41.293 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.178661078Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-16T08:46:41.294 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.179386026Z level=info msg="Migration successfully executed" id="Add is_service_account column to user" duration=724.623µs 2024-09-16T08:46:41.294 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.179737505Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-16T08:46:41.294 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.184932958Z level=info msg="Migration successfully executed" id="Update is_service_account column to nullable" duration=5.194056ms 2024-09-16T08:46:41.294 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.185287301Z level=info msg="Executing migration" id="Add uid column to user" 2024-09-16T08:46:41.294 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.186011595Z level=info msg="Migration successfully executed" id="Add uid column to user" duration=722.645µs 2024-09-16T08:46:41.294 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.186380619Z level=info msg="Executing migration" id="Update uid column values for users" 2024-09-16T08:46:41.295 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.186509518Z level=info msg="Migration successfully executed" id="Update uid column values for users" duration=130.015µs 2024-09-16T08:46:41.295 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.186878599Z level=info msg="Executing migration" id="Add unique index user_uid" 2024-09-16T08:46:41.295 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.187324519Z level=info msg="Migration successfully executed" id="Add unique index user_uid" duration=444.81µs 2024-09-16T08:46:41.295 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.187740573Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-16T08:46:41.295 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.18831989Z level=info msg="Migration successfully executed" id="create temp user table v1-7" duration=579.488µs 2024-09-16T08:46:41.295 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.188748923Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-16T08:46:41.296 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.189227745Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_email - v1-7" duration=478.579µs 2024-09-16T08:46:41.296 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.189593332Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-16T08:46:41.296 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.190065262Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_org_id - v1-7" duration=471.764µs 2024-09-16T08:46:41.296 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.190437025Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-16T08:46:41.297 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.190909509Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_code - v1-7" duration=472.264µs 2024-09-16T08:46:41.297 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.191281723Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-16T08:46:41.297 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.191748421Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_status - v1-7" duration=466.502µs 2024-09-16T08:46:41.297 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.192129017Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-16T08:46:41.297 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.192161762Z level=info msg="Migration successfully executed" id="Update temp_user table charset" duration=33.09µs 2024-09-16T08:46:41.297 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.192612543Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-16T08:46:41.297 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.193102472Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_email - v1" duration=489.761µs 2024-09-16T08:46:41.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.19348704Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-16T08:46:41.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.193970817Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_org_id - v1" duration=483.619µs 2024-09-16T08:46:41.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.194371184Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-16T08:46:41.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.194896519Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_code - v1" duration=525.18µs 2024-09-16T08:46:41.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.195259392Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-16T08:46:41.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.195707799Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_status - v1" duration=449.317µs 2024-09-16T08:46:41.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.196088427Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-16T08:46:41.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.19812614Z level=info msg="Migration successfully executed" id="Rename table temp_user to temp_user_tmp_qwerty - v1" duration=2.03743ms 2024-09-16T08:46:41.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.198884912Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-16T08:46:41.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.199446075Z level=info msg="Migration successfully executed" id="create temp_user v2" duration=559.377µs 2024-09-16T08:46:41.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.199825975Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-16T08:46:41.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.200309162Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_email - v2" duration=483.075µs 2024-09-16T08:46:41.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.200692816Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-16T08:46:41.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.201203482Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_org_id - v2" duration=510.988µs 2024-09-16T08:46:41.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.201589336Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-16T08:46:41.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.202106748Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_code - v2" duration=515.993µs 2024-09-16T08:46:41.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.20455349Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-16T08:46:41.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.20508568Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_status - v2" duration=531.925µs 2024-09-16T08:46:41.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.205474481Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-16T08:46:41.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.205777947Z level=info msg="Migration successfully executed" id="copy temp_user v1 to v2" duration=303.154µs 2024-09-16T08:46:41.301 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.20614088Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-16T08:46:41.301 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.206549259Z level=info msg="Migration successfully executed" id="drop temp_user_tmp_qwerty" duration=408.238µs 2024-09-16T08:46:41.301 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.206917683Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-16T08:46:41.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.207182733Z level=info msg="Migration successfully executed" id="Set created for temp users that will otherwise prematurely expire" duration=266.083µs 2024-09-16T08:46:41.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.207576297Z level=info msg="Executing migration" id="create star table" 2024-09-16T08:46:41.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.208033258Z level=info msg="Migration successfully executed" id="create star table" duration=456.666µs 2024-09-16T08:46:41.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.208384154Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-16T08:46:41.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.208882692Z level=info msg="Migration successfully executed" id="add unique index star.user_id_dashboard_id" duration=498.204µs 2024-09-16T08:46:41.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.209249691Z level=info msg="Executing migration" id="create org table v1" 2024-09-16T08:46:41.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.209749794Z level=info msg="Migration successfully executed" id="create org table v1" duration=499.625µs 2024-09-16T08:46:41.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.210134209Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-16T08:46:41.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.210615208Z level=info msg="Migration successfully executed" id="create index UQE_org_name - v1" duration=480.729µs 2024-09-16T08:46:41.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.210987928Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-16T08:46:41.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.211452056Z level=info msg="Migration successfully executed" id="create org_user table v1" duration=459.884µs 2024-09-16T08:46:41.303 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.21183963Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-16T08:46:41.303 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.212385573Z level=info msg="Migration successfully executed" id="create index IDX_org_user_org_id - v1" duration=546.541µs 2024-09-16T08:46:41.303 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.214891339Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-16T08:46:41.303 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.215395089Z level=info msg="Migration successfully executed" id="create index UQE_org_user_org_id_user_id - v1" duration=503.595µs 2024-09-16T08:46:41.303 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.217931297Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-16T08:46:41.303 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.218488415Z level=info msg="Migration successfully executed" id="create index IDX_org_user_user_id - v1" duration=559.242µs 2024-09-16T08:46:41.303 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.220657679Z level=info msg="Executing migration" id="Update org table charset" 2024-09-16T08:46:41.303 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.220690184Z level=info msg="Migration successfully executed" id="Update org table charset" duration=32.995µs 2024-09-16T08:46:41.303 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.224258011Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-16T08:46:41.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.224290218Z level=info msg="Migration successfully executed" id="Update org_user table charset" duration=32.728µs 2024-09-16T08:46:41.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.22726848Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-16T08:46:41.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.227518805Z level=info msg="Migration successfully executed" id="Migrate all Read Only Viewers to Viewers" duration=252.532µs 2024-09-16T08:46:41.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.229681871Z level=info msg="Executing migration" id="create dashboard table" 2024-09-16T08:46:41.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.230199392Z level=info msg="Migration successfully executed" id="create dashboard table" duration=518.037µs 2024-09-16T08:46:41.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.232340043Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-16T08:46:41.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.232872056Z level=info msg="Migration successfully executed" id="add index dashboard.account_id" duration=531.586µs 2024-09-16T08:46:41.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.234928312Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-16T08:46:41.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.235465473Z level=info msg="Migration successfully executed" id="add unique index dashboard_account_id_slug" duration=536.964µs 2024-09-16T08:46:41.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.237585446Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-16T08:46:41.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.238048158Z level=info msg="Migration successfully executed" id="create dashboard_tag table" duration=462.537µs 2024-09-16T08:46:41.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.24031155Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-16T08:46:41.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.240814248Z level=info msg="Migration successfully executed" id="add unique index dashboard_tag.dasboard_id_term" duration=501.234µs 2024-09-16T08:46:41.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.243028954Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-16T08:46:41.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.243472694Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" duration=443.699µs 2024-09-16T08:46:41.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.245601783Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-16T08:46:41.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.249062134Z level=info msg="Migration successfully executed" id="Rename table dashboard to dashboard_v1 - v1" duration=3.459802ms 2024-09-16T08:46:41.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.249428955Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-16T08:46:41.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.249923406Z level=info msg="Migration successfully executed" id="create dashboard v2" duration=494.255µs 2024-09-16T08:46:41.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.252186581Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-16T08:46:41.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.252657838Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_org_id - v2" duration=471.032µs 2024-09-16T08:46:41.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.254910126Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-16T08:46:41.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.255401744Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_org_id_slug - v2" duration=491.424µs 2024-09-16T08:46:41.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.257598879Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-16T08:46:41.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.257871749Z level=info msg="Migration successfully executed" id="copy dashboard v1 to v2" duration=271.678µs 2024-09-16T08:46:41.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.260589844Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-16T08:46:41.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.261350018Z level=info msg="Migration successfully executed" id="drop table dashboard_v1" duration=762.425µs 2024-09-16T08:46:41.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.263038748Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-16T08:46:41.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.263079009Z level=info msg="Migration successfully executed" id="alter dashboard.data to mediumtext v1" duration=39.573µs 2024-09-16T08:46:41.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.265601083Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-16T08:46:41.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.266690865Z level=info msg="Migration successfully executed" id="Add column updated_by in dashboard - v2" duration=1.08958ms 2024-09-16T08:46:41.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.268436172Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-16T08:46:41.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.269503912Z level=info msg="Migration successfully executed" id="Add column created_by in dashboard - v2" duration=1.067558ms 2024-09-16T08:46:41.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.270929562Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-16T08:46:41.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.271988175Z level=info msg="Migration successfully executed" id="Add column gnetId in dashboard" duration=1.057935ms 2024-09-16T08:46:41.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.273855045Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-16T08:46:41.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.274343284Z level=info msg="Migration successfully executed" id="Add index for gnetId in dashboard" duration=488.16µs 2024-09-16T08:46:41.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.274701258Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-16T08:46:41.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.275797169Z level=info msg="Migration successfully executed" id="Add column plugin_id in dashboard" duration=1.095544ms 2024-09-16T08:46:41.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.276162917Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-16T08:46:41.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.276662531Z level=info msg="Migration successfully executed" id="Add index for plugin_id in dashboard" duration=499.595µs 2024-09-16T08:46:41.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.277057471Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-16T08:46:41.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.27753746Z level=info msg="Migration successfully executed" id="Add index for dashboard_id in dashboard_tag" duration=479.681µs 2024-09-16T08:46:41.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.277959482Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-16T08:46:41.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.277983409Z level=info msg="Migration successfully executed" id="Update dashboard table charset" duration=24.968µs 2024-09-16T08:46:41.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.278758203Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-16T08:46:41.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.278786969Z level=info msg="Migration successfully executed" id="Update dashboard_tag table charset" duration=29.5µs 2024-09-16T08:46:41.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.279337074Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-16T08:46:41.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.281240407Z level=info msg="Migration successfully executed" id="Add column folder_id in dashboard" duration=1.902044ms 2024-09-16T08:46:41.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.28169186Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-16T08:46:41.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.283577768Z level=info msg="Migration successfully executed" id="Add column isFolder in dashboard" duration=1.885592ms 2024-09-16T08:46:41.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.284045018Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-16T08:46:41.528 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.285618763Z level=info msg="Migration successfully executed" id="Add column has_acl in dashboard" duration=1.57421ms 2024-09-16T08:46:41.528 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.285974478Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-16T08:46:41.528 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.287124308Z level=info msg="Migration successfully executed" id="Add column uid in dashboard" duration=1.149842ms 2024-09-16T08:46:41.528 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.287470018Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-16T08:46:41.529 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.287609222Z level=info msg="Migration successfully executed" id="Update uid column values in dashboard" duration=139.517µs 2024-09-16T08:46:41.529 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.288321863Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-16T08:46:41.529 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.288802812Z level=info msg="Migration successfully executed" id="Add unique index dashboard_org_id_uid" duration=481.792µs 2024-09-16T08:46:41.529 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.289178357Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-16T08:46:41.529 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.289632714Z level=info msg="Migration successfully executed" id="Remove unique index org_id_slug" duration=454.406µs 2024-09-16T08:46:41.529 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.290023876Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-16T08:46:41.529 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.290039509Z level=info msg="Migration successfully executed" id="Update dashboard title length" duration=16.077µs 2024-09-16T08:46:41.529 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.290906742Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-16T08:46:41.529 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.2913879Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_title_folder_id" duration=480.898µs 2024-09-16T08:46:41.529 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.291759742Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-16T08:46:41.529 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.292207064Z level=info msg="Migration successfully executed" id="create dashboard_provisioning" duration=447.833µs 2024-09-16T08:46:41.529 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.292619095Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-16T08:46:41.530 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.295683289Z level=info msg="Migration successfully executed" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" duration=3.06399ms 2024-09-16T08:46:41.530 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.296042572Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-16T08:46:41.530 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.296504735Z level=info msg="Migration successfully executed" id="create dashboard_provisioning v2" duration=461.972µs 2024-09-16T08:46:41.530 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.296893884Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-16T08:46:41.530 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.297369283Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_provisioning_dashboard_id - v2" duration=476.103µs 2024-09-16T08:46:41.530 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.297740722Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-16T08:46:41.530 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.298226917Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" duration=485.82µs 2024-09-16T08:46:41.530 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.298596442Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-16T08:46:41.530 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.29881872Z level=info msg="Migration successfully executed" id="copy dashboard_provisioning v1 to v2" duration=222.334µs 2024-09-16T08:46:41.530 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.299457036Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-16T08:46:41.530 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.299834475Z level=info msg="Migration successfully executed" id="drop dashboard_provisioning_tmp_qwerty" duration=377.238µs 2024-09-16T08:46:41.530 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.300341281Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-16T08:46:41.530 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.301546803Z level=info msg="Migration successfully executed" id="Add check_sum column" duration=1.205153ms 2024-09-16T08:46:41.531 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.301892827Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-16T08:46:41.531 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.302350343Z level=info msg="Migration successfully executed" id="Add index for dashboard_title" duration=458.285µs 2024-09-16T08:46:41.531 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.302771588Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-16T08:46:41.531 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.302879951Z level=info msg="Migration successfully executed" id="delete tags for deleted dashboards" duration=108.423µs 2024-09-16T08:46:41.531 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.303647938Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-16T08:46:41.531 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.303747506Z level=info msg="Migration successfully executed" id="delete stars for deleted dashboards" duration=99.608µs 2024-09-16T08:46:41.533 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.30451221Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-16T08:46:41.533 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.305001902Z level=info msg="Migration successfully executed" id="Add index for dashboard_is_folder" duration=489.313µs 2024-09-16T08:46:41.533 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.305377391Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-16T08:46:41.533 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.306634041Z level=info msg="Migration successfully executed" id="Add isPublic for dashboard" duration=1.25624ms 2024-09-16T08:46:41.533 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.306995962Z level=info msg="Executing migration" id="create data_source table" 2024-09-16T08:46:41.533 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.307586628Z level=info msg="Migration successfully executed" id="create data_source table" duration=590.479µs 2024-09-16T08:46:41.533 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.307975367Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-16T08:46:41.533 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.308465439Z level=info msg="Migration successfully executed" id="add index data_source.account_id" duration=489.968µs 2024-09-16T08:46:41.533 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.308881738Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-16T08:46:41.536 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.309360824Z level=info msg="Migration successfully executed" id="add unique index data_source.account_id_name" duration=477.94µs 2024-09-16T08:46:41.536 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.309738483Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-16T08:46:41.536 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.310199386Z level=info msg="Migration successfully executed" id="drop index IDX_data_source_account_id - v1" duration=460.71µs 2024-09-16T08:46:41.536 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.310640558Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-16T08:46:41.536 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.311101945Z level=info msg="Migration successfully executed" id="drop index UQE_data_source_account_id_name - v1" duration=462.232µs 2024-09-16T08:46:41.536 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.311541071Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-16T08:46:41.536 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.314899302Z level=info msg="Migration successfully executed" id="Rename table data_source to data_source_v1 - v1" duration=3.358022ms 2024-09-16T08:46:41.536 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.315276178Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-16T08:46:41.537 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.315880905Z level=info msg="Migration successfully executed" id="create data_source table v2" duration=604.512µs 2024-09-16T08:46:41.537 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.316228045Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-16T08:46:41.537 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.31673752Z level=info msg="Migration successfully executed" id="create index IDX_data_source_org_id - v2" duration=509.133µs 2024-09-16T08:46:41.537 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.317123183Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-16T08:46:41.537 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.317645944Z level=info msg="Migration successfully executed" id="create index UQE_data_source_org_id_name - v2" duration=522.425µs 2024-09-16T08:46:41.537 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.319986043Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-16T08:46:41.537 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.320386388Z level=info msg="Migration successfully executed" id="Drop old table data_source_v1 #2" duration=400.215µs 2024-09-16T08:46:41.537 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.322857078Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-16T08:46:41.537 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.324274937Z level=info msg="Migration successfully executed" id="Add column with_credentials" duration=1.417568ms 2024-09-16T08:46:41.537 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.325835883Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-16T08:46:41.537 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.327213437Z level=info msg="Migration successfully executed" id="Add secure json data column" duration=1.37714ms 2024-09-16T08:46:41.537 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.327572614Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-16T08:46:41.537 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.327588848Z level=info msg="Migration successfully executed" id="Update data_source table charset" duration=15.933µs 2024-09-16T08:46:41.538 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.330329998Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-16T08:46:41.538 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.330520649Z level=info msg="Migration successfully executed" id="Update initial version to 1" duration=192.933µs 2024-09-16T08:46:41.538 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.33262825Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-16T08:46:41.538 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.334035769Z level=info msg="Migration successfully executed" id="Add read_only data column" duration=1.407909ms 2024-09-16T08:46:41.538 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.33513457Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-16T08:46:41.538 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.335277104Z level=info msg="Migration successfully executed" id="Migrate logging ds to loki ds" duration=142.551µs 2024-09-16T08:46:41.538 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.338028474Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-16T08:46:41.538 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.338192047Z level=info msg="Migration successfully executed" id="Update json_data with nulls" duration=165.735µs 2024-09-16T08:46:41.538 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.34054484Z level=info msg="Executing migration" id="Add uid column" 2024-09-16T08:46:41.538 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.341983314Z level=info msg="Migration successfully executed" id="Add uid column" duration=1.437852ms 2024-09-16T08:46:41.538 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.343259661Z level=info msg="Executing migration" id="Update uid value" 2024-09-16T08:46:41.538 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.343402991Z level=info msg="Migration successfully executed" id="Update uid value" duration=143.656µs 2024-09-16T08:46:41.539 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.345907743Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-16T08:46:41.539 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.346446314Z level=info msg="Migration successfully executed" id="Add unique index datasource_org_id_uid" duration=538.229µs 2024-09-16T08:46:41.539 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.348675143Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-16T08:46:41.539 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.349294491Z level=info msg="Migration successfully executed" id="add unique index datasource_org_id_is_default" duration=616.584µs 2024-09-16T08:46:41.539 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.349658742Z level=info msg="Executing migration" id="create api_key table" 2024-09-16T08:46:41.539 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.35020493Z level=info msg="Migration successfully executed" id="create api_key table" duration=542.116µs 2024-09-16T08:46:41.539 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.352250522Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-16T08:46:41.539 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.352778676Z level=info msg="Migration successfully executed" id="add index api_key.account_id" duration=527.888µs 2024-09-16T08:46:41.539 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.354822231Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-16T08:46:41.539 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.35531252Z level=info msg="Migration successfully executed" id="add index api_key.key" duration=491.733µs 2024-09-16T08:46:41.539 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.357453828Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-16T08:46:41.539 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.358055741Z level=info msg="Migration successfully executed" id="add index api_key.account_id_name" duration=603.659µs 2024-09-16T08:46:41.539 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.358438234Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-16T08:46:41.539 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.358921751Z level=info msg="Migration successfully executed" id="drop index IDX_api_key_account_id - v1" duration=483.238µs 2024-09-16T08:46:41.540 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.361153043Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-16T08:46:41.540 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.361680582Z level=info msg="Migration successfully executed" id="drop index UQE_api_key_key - v1" duration=529.754µs 2024-09-16T08:46:41.540 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.362025245Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-16T08:46:41.540 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.362492285Z level=info msg="Migration successfully executed" id="drop index UQE_api_key_account_id_name - v1" duration=466.819µs 2024-09-16T08:46:41.540 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.362859624Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-16T08:46:41.540 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.366478225Z level=info msg="Migration successfully executed" id="Rename table api_key to api_key_v1 - v1" duration=3.618257ms 2024-09-16T08:46:41.540 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.366843128Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-16T08:46:41.540 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.367329811Z level=info msg="Migration successfully executed" id="create api_key table v2" duration=485.284µs 2024-09-16T08:46:41.540 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.367683212Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-16T08:46:41.540 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.368195671Z level=info msg="Migration successfully executed" id="create index IDX_api_key_org_id - v2" duration=512.124µs 2024-09-16T08:46:41.540 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.368543827Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-16T08:46:41.540 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.369044264Z level=info msg="Migration successfully executed" id="create index UQE_api_key_key - v2" duration=500.213µs 2024-09-16T08:46:41.540 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.369388256Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-16T08:46:41.541 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.369875619Z level=info msg="Migration successfully executed" id="create index UQE_api_key_org_id_name - v2" duration=487.202µs 2024-09-16T08:46:41.541 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.370219019Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-16T08:46:41.541 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.370454482Z level=info msg="Migration successfully executed" id="copy api_key v1 to v2" duration=235.293µs 2024-09-16T08:46:41.541 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.370957278Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-16T08:46:41.541 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.371339406Z level=info msg="Migration successfully executed" id="Drop old table api_key_v1" duration=382.075µs 2024-09-16T08:46:41.541 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.371749831Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-16T08:46:41.541 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.371772046Z level=info msg="Migration successfully executed" id="Update api_key table charset" duration=22.764µs 2024-09-16T08:46:41.541 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.372658943Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-16T08:46:41.541 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.37414407Z level=info msg="Migration successfully executed" id="Add expires to api_key table" duration=1.484768ms 2024-09-16T08:46:41.541 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.374491717Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-16T08:46:41.541 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.375958257Z level=info msg="Migration successfully executed" id="Add service account foreign key" duration=1.466178ms 2024-09-16T08:46:41.541 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.376292787Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-16T08:46:41.542 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.376394658Z level=info msg="Migration successfully executed" id="set service account foreign key to nil if 0" duration=100.969µs 2024-09-16T08:46:41.542 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.377235341Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-16T08:46:41.542 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.378887451Z level=info msg="Migration successfully executed" id="Add last_used_at to api_key table" duration=1.654087ms 2024-09-16T08:46:41.542 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.379242066Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-16T08:46:41.542 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.380959398Z level=info msg="Migration successfully executed" id="Add is_revoked column to api_key table" duration=1.717167ms 2024-09-16T08:46:41.542 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.381322516Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-16T08:46:41.542 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.381852786Z level=info msg="Migration successfully executed" id="create dashboard_snapshot table v4" duration=529.954µs 2024-09-16T08:46:41.542 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.382200208Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-16T08:46:41.542 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.382588826Z level=info msg="Migration successfully executed" id="drop table dashboard_snapshot_v4 #1" duration=387.985µs 2024-09-16T08:46:41.542 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.382960614Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-16T08:46:41.542 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.383512823Z level=info msg="Migration successfully executed" id="create dashboard_snapshot table v5 #2" duration=551.979µs 2024-09-16T08:46:41.542 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.383849642Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-16T08:46:41.542 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.384385371Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_snapshot_key - v5" duration=535.593µs 2024-09-16T08:46:41.543 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.384742791Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-16T08:46:41.543 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.38531894Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_snapshot_delete_key - v5" duration=575.896µs 2024-09-16T08:46:41.543 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.385677312Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-16T08:46:41.543 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.386186838Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_snapshot_user_id - v5" duration=509.258µs 2024-09-16T08:46:41.543 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.386523506Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-16T08:46:41.543 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.386562741Z level=info msg="Migration successfully executed" id="alter dashboard_snapshot to mediumtext v2" duration=39.63µs 2024-09-16T08:46:41.543 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.387482562Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-16T08:46:41.543 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.387522005Z level=info msg="Migration successfully executed" id="Update dashboard_snapshot table charset" duration=40.966µs 2024-09-16T08:46:41.543 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.388143594Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-16T08:46:41.543 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.389782301Z level=info msg="Migration successfully executed" id="Add column external_delete_url to dashboard_snapshots table" duration=1.638462ms 2024-09-16T08:46:41.543 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.390147633Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-16T08:46:41.543 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.391747622Z level=info msg="Migration successfully executed" id="Add encrypted dashboard json column" duration=1.599873ms 2024-09-16T08:46:41.543 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.392132226Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-16T08:46:41.544 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.392172842Z level=info msg="Migration successfully executed" id="Change dashboard_encrypted column to MEDIUMBLOB" duration=40.853µs 2024-09-16T08:46:41.544 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.392908553Z level=info msg="Executing migration" id="create quota table v1" 2024-09-16T08:46:41.544 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.393388863Z level=info msg="Migration successfully executed" id="create quota table v1" duration=480.86µs 2024-09-16T08:46:41.544 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.3937537Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-16T08:46:41.544 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.394263504Z level=info msg="Migration successfully executed" id="create index UQE_quota_org_id_user_id_target - v1" duration=509.6µs 2024-09-16T08:46:41.544 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.394607105Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-16T08:46:41.544 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.394622091Z level=info msg="Migration successfully executed" id="Update quota table charset" duration=15.588µs 2024-09-16T08:46:41.544 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.395447194Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-16T08:46:41.544 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.395941572Z level=info msg="Migration successfully executed" id="create plugin_setting table" duration=495.061µs 2024-09-16T08:46:41.544 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.396277721Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-16T08:46:41.544 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.39681109Z level=info msg="Migration successfully executed" id="create index UQE_plugin_setting_org_id_plugin_id - v1" duration=533.05µs 2024-09-16T08:46:41.545 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.397143289Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-16T08:46:41.545 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.398737672Z level=info msg="Migration successfully executed" id="Add column plugin_version to plugin_settings" duration=1.594096ms 2024-09-16T08:46:41.545 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.39909867Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-16T08:46:41.545 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.399114192Z level=info msg="Migration successfully executed" id="Update plugin_setting table charset" duration=16.009µs 2024-09-16T08:46:41.545 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.399920227Z level=info msg="Executing migration" id="create session table" 2024-09-16T08:46:41.545 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.400481041Z level=info msg="Migration successfully executed" id="create session table" duration=559.426µs 2024-09-16T08:46:41.545 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.400866673Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-16T08:46:41.545 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.400926407Z level=info msg="Migration successfully executed" id="Drop old table playlist table" duration=59.971µs 2024-09-16T08:46:41.545 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.401625269Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-16T08:46:41.545 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.401695399Z level=info msg="Migration successfully executed" id="Drop old table playlist_item table" duration=70.176µs 2024-09-16T08:46:41.545 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.40445886Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-16T08:46:41.545 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.404970209Z level=info msg="Migration successfully executed" id="create playlist table v2" duration=510.984µs 2024-09-16T08:46:41.545 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.407264282Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-16T08:46:41.546 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.407789341Z level=info msg="Migration successfully executed" id="create playlist item table v2" duration=524.881µs 2024-09-16T08:46:41.546 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.41005031Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-16T08:46:41.546 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.410065163Z level=info msg="Migration successfully executed" id="Update playlist table charset" duration=15.464µs 2024-09-16T08:46:41.546 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.41277987Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-16T08:46:41.546 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.412817263Z level=info msg="Migration successfully executed" id="Update playlist_item table charset" duration=37.932µs 2024-09-16T08:46:41.546 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.415704406Z level=info msg="Executing migration" id="Add playlist column created_at" 2024-09-16T08:46:41.546 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.417643004Z level=info msg="Migration successfully executed" id="Add playlist column created_at" duration=1.940668ms 2024-09-16T08:46:41.546 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.418283992Z level=info msg="Executing migration" id="Add playlist column updated_at" 2024-09-16T08:46:41.546 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.420173964Z level=info msg="Migration successfully executed" id="Add playlist column updated_at" duration=1.888346ms 2024-09-16T08:46:41.546 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.421267689Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-16T08:46:41.546 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.421424823Z level=info msg="Migration successfully executed" id="drop preferences table v2" duration=157.329µs 2024-09-16T08:46:41.546 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.423928793Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-16T08:46:41.546 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.423985905Z level=info msg="Migration successfully executed" id="drop preferences table v3" duration=57.214µs 2024-09-16T08:46:41.547 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.426817242Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-16T08:46:41.547 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.427345395Z level=info msg="Migration successfully executed" id="create preferences table v3" duration=528.766µs 2024-09-16T08:46:41.547 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.429569872Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-16T08:46:41.547 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.429584593Z level=info msg="Migration successfully executed" id="Update preferences table charset" duration=15.453µs 2024-09-16T08:46:41.547 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.43218051Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-16T08:46:41.547 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.433955989Z level=info msg="Migration successfully executed" id="Add column team_id in preferences" duration=1.775039ms 2024-09-16T08:46:41.547 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.434911878Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-16T08:46:41.547 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.435024318Z level=info msg="Migration successfully executed" id="Update team_id column values in preferences" duration=112.61µs 2024-09-16T08:46:41.547 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.437603459Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-16T08:46:41.547 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.439424698Z level=info msg="Migration successfully executed" id="Add column week_start in preferences" duration=1.821764ms 2024-09-16T08:46:41.547 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.440603062Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-16T08:46:41.547 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.442628638Z level=info msg="Migration successfully executed" id="Add column preferences.json_data" duration=2.028482ms 2024-09-16T08:46:41.548 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.443224184Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-16T08:46:41.548 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.443284173Z level=info msg="Migration successfully executed" id="alter preferences.json_data to mediumtext v1" duration=60.318µs 2024-09-16T08:46:41.548 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.446144624Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-16T08:46:41.548 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.446795825Z level=info msg="Migration successfully executed" id="Add preferences index org_id" duration=651.218µs 2024-09-16T08:46:41.548 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.44884814Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-16T08:46:41.548 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.449401736Z level=info msg="Migration successfully executed" id="Add preferences index user_id" duration=553.59µs 2024-09-16T08:46:41.548 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.451419298Z level=info msg="Executing migration" id="create alert table v1" 2024-09-16T08:46:41.548 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.452267968Z level=info msg="Migration successfully executed" id="create alert table v1" duration=851.066µs 2024-09-16T08:46:41.548 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.453697438Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-16T08:46:41.548 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.454331195Z level=info msg="Migration successfully executed" id="add index alert org_id & id " duration=633.397µs 2024-09-16T08:46:41.548 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.456458583Z level=info msg="Executing migration" id="add index alert state" 2024-09-16T08:46:41.548 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.457054522Z level=info msg="Migration successfully executed" id="add index alert state" duration=595.738µs 2024-09-16T08:46:41.549 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.458843146Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-16T08:46:41.549 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.459384017Z level=info msg="Migration successfully executed" id="add index alert dashboard_id" duration=540.722µs 2024-09-16T08:46:41.549 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.461576839Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-16T08:46:41.549 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.462104185Z level=info msg="Migration successfully executed" id="Create alert_rule_tag table v1" duration=527.204µs 2024-09-16T08:46:41.549 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.463956437Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-16T08:46:41.549 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.46452515Z level=info msg="Migration successfully executed" id="Add unique index alert_rule_tag.alert_id_tag_id" duration=567.321µs 2024-09-16T08:46:41.549 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.466756201Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-16T08:46:41.549 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.467365021Z level=info msg="Migration successfully executed" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" duration=611.123µs 2024-09-16T08:46:41.549 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.469221569Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-16T08:46:41.549 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.474978543Z level=info msg="Migration successfully executed" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" duration=5.756443ms 2024-09-16T08:46:41.549 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.475382659Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-16T08:46:41.549 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.475853278Z level=info msg="Migration successfully executed" id="Create alert_rule_tag table v2" duration=470.449µs 2024-09-16T08:46:41.550 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.477845594Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-16T08:46:41.550 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.478381317Z level=info msg="Migration successfully executed" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" duration=535.589µs 2024-09-16T08:46:41.550 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.480268184Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-16T08:46:41.550 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.480485964Z level=info msg="Migration successfully executed" id="copy alert_rule_tag v1 to v2" duration=218.601µs 2024-09-16T08:46:41.550 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.483051658Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-16T08:46:41.550 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.483423207Z level=info msg="Migration successfully executed" id="drop table alert_rule_tag_v1" duration=371.492µs 2024-09-16T08:46:41.550 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.485315861Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-16T08:46:41.550 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.485818434Z level=info msg="Migration successfully executed" id="create alert_notification table v1" duration=502.403µs 2024-09-16T08:46:41.550 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.488059127Z level=info msg="Executing migration" id="Add column is_default" 2024-09-16T08:46:41.550 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.490123874Z level=info msg="Migration successfully executed" id="Add column is_default" duration=2.066017ms 2024-09-16T08:46:41.550 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.490478421Z level=info msg="Executing migration" id="Add column frequency" 2024-09-16T08:46:41.550 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.492614058Z level=info msg="Migration successfully executed" id="Add column frequency" duration=2.135374ms 2024-09-16T08:46:41.551 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.49309537Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-16T08:46:41.551 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.495439637Z level=info msg="Migration successfully executed" id="Add column send_reminder" duration=2.344023ms 2024-09-16T08:46:41.551 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.4958083Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-16T08:46:41.551 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.497907977Z level=info msg="Migration successfully executed" id="Add column disable_resolve_message" duration=2.099351ms 2024-09-16T08:46:41.551 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.498361284Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-16T08:46:41.551 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.498904424Z level=info msg="Migration successfully executed" id="add index alert_notification org_id & name" duration=542.841µs 2024-09-16T08:46:41.551 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.499251282Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-16T08:46:41.551 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.49926973Z level=info msg="Migration successfully executed" id="Update alert table charset" duration=18.623µs 2024-09-16T08:46:41.551 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.50002776Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-16T08:46:41.551 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.500047749Z level=info msg="Migration successfully executed" id="Update alert_notification table charset" duration=18.696µs 2024-09-16T08:46:41.551 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.500984926Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-16T08:46:41.551 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.501548813Z level=info msg="Migration successfully executed" id="create notification_journal table v1" duration=567.29µs 2024-09-16T08:46:41.551 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.501898777Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-16T08:46:41.552 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.502431994Z level=info msg="Migration successfully executed" id="add index notification_journal org_id & alert_id & notifier_id" duration=533.048µs 2024-09-16T08:46:41.552 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.502815353Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-16T08:46:41.552 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.503340042Z level=info msg="Migration successfully executed" id="drop alert_notification_journal" duration=524.558µs 2024-09-16T08:46:41.552 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.503715558Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-16T08:46:41.552 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.504244143Z level=info msg="Migration successfully executed" id="create alert_notification_state table v1" duration=528.539µs 2024-09-16T08:46:41.552 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.504608152Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-16T08:46:41.552 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.50515285Z level=info msg="Migration successfully executed" id="add index alert_notification_state org_id & alert_id & notifier_id" duration=544.503µs 2024-09-16T08:46:41.552 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.505510373Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-16T08:46:41.552 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.507628401Z level=info msg="Migration successfully executed" id="Add for to alert table" duration=2.117767ms 2024-09-16T08:46:41.552 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.508004551Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-16T08:46:41.553 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.510218719Z level=info msg="Migration successfully executed" id="Add column uid in alert_notification" duration=2.21394ms 2024-09-16T08:46:41.553 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.510578749Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-16T08:46:41.553 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.510700614Z level=info msg="Migration successfully executed" id="Update uid column values in alert_notification" duration=121.758µs 2024-09-16T08:46:41.553 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.51137604Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-16T08:46:41.553 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.511919463Z level=info msg="Migration successfully executed" id="Add unique index alert_notification_org_id_uid" duration=543.056µs 2024-09-16T08:46:41.553 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.512323185Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-16T08:46:41.553 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.512863601Z level=info msg="Migration successfully executed" id="Remove unique index org_id_name" duration=540.131µs 2024-09-16T08:46:41.553 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.513231393Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-16T08:46:41.554 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.515368678Z level=info msg="Migration successfully executed" id="Add column secure_settings in alert_notification" duration=2.136816ms 2024-09-16T08:46:41.554 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.515727424Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-16T08:46:41.554 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.515786734Z level=info msg="Migration successfully executed" id="alter alert.settings to mediumtext" duration=59.483µs 2024-09-16T08:46:41.554 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.516494985Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-16T08:46:41.554 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.517049179Z level=info msg="Migration successfully executed" id="Add non-unique index alert_notification_state_alert_id" duration=553.848µs 2024-09-16T08:46:41.554 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.517421439Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-16T08:46:41.554 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.518013365Z level=info msg="Migration successfully executed" id="Add non-unique index alert_rule_tag_alert_id" duration=593.336µs 2024-09-16T08:46:41.554 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.518410441Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-16T08:46:41.554 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.518481753Z level=info msg="Migration successfully executed" id="Drop old annotation table v4" duration=69.876µs 2024-09-16T08:46:41.554 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.519216024Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-16T08:46:41.554 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.519790389Z level=info msg="Migration successfully executed" id="create annotation table v5" duration=574.023µs 2024-09-16T08:46:41.554 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.520156218Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-16T08:46:41.554 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.520683509Z level=info msg="Migration successfully executed" id="add index annotation 0 v3" duration=526.991µs 2024-09-16T08:46:41.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.521043264Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-16T08:46:41.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.521560805Z level=info msg="Migration successfully executed" id="add index annotation 1 v3" duration=517.316µs 2024-09-16T08:46:41.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.521911995Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-16T08:46:41.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.522436042Z level=info msg="Migration successfully executed" id="add index annotation 2 v3" duration=523.8µs 2024-09-16T08:46:41.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.522795307Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-16T08:46:41.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.523411715Z level=info msg="Migration successfully executed" id="add index annotation 3 v3" duration=616.149µs 2024-09-16T08:46:41.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.523816746Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-16T08:46:41.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.524418531Z level=info msg="Migration successfully executed" id="add index annotation 4 v3" duration=601.428µs 2024-09-16T08:46:41.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.524798509Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-16T08:46:41.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.524817703Z level=info msg="Migration successfully executed" id="Update annotation table charset" duration=19.294µs 2024-09-16T08:46:41.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.525574872Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-16T08:46:41.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.528005157Z level=info msg="Migration successfully executed" id="Add column region_id to annotation table" duration=2.430123ms 2024-09-16T08:46:41.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.528406724Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-16T08:46:41.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.528948795Z level=info msg="Migration successfully executed" id="Drop category_id index" duration=542.094µs 2024-09-16T08:46:41.556 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.529299814Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-16T08:46:41.556 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.531595506Z level=info msg="Migration successfully executed" id="Add column tags to annotation table" duration=2.295877ms 2024-09-16T08:46:41.556 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.531955955Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-16T08:46:41.556 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.53239183Z level=info msg="Migration successfully executed" id="Create annotation_tag table v2" duration=435.507µs 2024-09-16T08:46:41.556 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.532756641Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-16T08:46:41.556 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.533335555Z level=info msg="Migration successfully executed" id="Add unique index annotation_tag.annotation_id_tag_id" duration=578.596µs 2024-09-16T08:46:41.556 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.533755514Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-16T08:46:41.556 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.534581918Z level=info msg="Migration successfully executed" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" duration=826.959µs 2024-09-16T08:46:41.556 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.535053317Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-16T08:46:41.778 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.54176736Z level=info msg="Migration successfully executed" id="Rename table annotation_tag to annotation_tag_v2 - v2" duration=6.714289ms 2024-09-16T08:46:41.778 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.542122742Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-16T08:46:41.778 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.542571845Z level=info msg="Migration successfully executed" id="Create annotation_tag table v3" duration=448.51µs 2024-09-16T08:46:41.778 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.54291574Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-16T08:46:41.779 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.543450488Z level=info msg="Migration successfully executed" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" duration=534.627µs 2024-09-16T08:46:41.779 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.543795822Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-16T08:46:41.779 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.543984997Z level=info msg="Migration successfully executed" id="copy annotation_tag v2 to v3" duration=189.099µs 2024-09-16T08:46:41.779 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.544569362Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-16T08:46:41.779 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.544934578Z level=info msg="Migration successfully executed" id="drop table annotation_tag_v2" duration=362.305µs 2024-09-16T08:46:41.779 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.547218382Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-16T08:46:41.779 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.547326594Z level=info msg="Migration successfully executed" id="Update alert annotations and set TEXT to empty" duration=108.312µs 2024-09-16T08:46:41.779 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.550053223Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-16T08:46:41.779 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.552414439Z level=info msg="Migration successfully executed" id="Add created time to annotation table" duration=2.362599ms 2024-09-16T08:46:41.779 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.552788199Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-16T08:46:41.779 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.555094457Z level=info msg="Migration successfully executed" id="Add updated time to annotation table" duration=2.305952ms 2024-09-16T08:46:41.779 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.555487904Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-16T08:46:41.779 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.556042329Z level=info msg="Migration successfully executed" id="Add index for created in annotation table" duration=554.215µs 2024-09-16T08:46:41.780 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.558413489Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-16T08:46:41.780 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.558954761Z level=info msg="Migration successfully executed" id="Add index for updated in annotation table" duration=540.969µs 2024-09-16T08:46:41.780 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.559313184Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-16T08:46:41.780 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.559469282Z level=info msg="Migration successfully executed" id="Convert existing annotations from seconds to milliseconds" duration=156.09µs 2024-09-16T08:46:41.780 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.56031064Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-16T08:46:41.781 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.56283533Z level=info msg="Migration successfully executed" id="Add epoch_end column" duration=2.525174ms 2024-09-16T08:46:41.781 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.563205355Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-16T08:46:41.781 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.563743269Z level=info msg="Migration successfully executed" id="Add index for epoch_end" duration=537.633µs 2024-09-16T08:46:41.781 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.564124466Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-16T08:46:41.781 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.564244292Z level=info msg="Migration successfully executed" id="Make epoch_end the same as epoch" duration=119.753µs 2024-09-16T08:46:41.781 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.564951379Z level=info msg="Executing migration" id="Move region to single row" 2024-09-16T08:46:41.781 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.565156783Z level=info msg="Migration successfully executed" id="Move region to single row" duration=205.461µs 2024-09-16T08:46:41.781 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.565898427Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-16T08:46:41.781 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.566419445Z level=info msg="Migration successfully executed" id="Remove index org_id_epoch from annotation table" duration=520.83µs 2024-09-16T08:46:41.781 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.566767622Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-16T08:46:41.781 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.567294113Z level=info msg="Migration successfully executed" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" duration=526.557µs 2024-09-16T08:46:41.781 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.56764735Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-16T08:46:41.782 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.568204671Z level=info msg="Migration successfully executed" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" duration=557.037µs 2024-09-16T08:46:41.782 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.568557932Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-16T08:46:41.782 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.569108718Z level=info msg="Migration successfully executed" id="Add index for org_id_epoch_end_epoch on annotation table" duration=549.089µs 2024-09-16T08:46:41.782 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.569456234Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-16T08:46:41.782 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.569979509Z level=info msg="Migration successfully executed" id="Remove index org_id_epoch_epoch_end from annotation table" duration=522.94µs 2024-09-16T08:46:41.782 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.570347357Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-16T08:46:41.782 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.57088539Z level=info msg="Migration successfully executed" id="Add index for alert_id on annotation table" duration=537.156µs 2024-09-16T08:46:41.782 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.571231269Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-16T08:46:41.782 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.571269646Z level=info msg="Migration successfully executed" id="Increase tags column to length 4096" duration=38.685µs 2024-09-16T08:46:41.782 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.572084251Z level=info msg="Executing migration" id="create test_data table" 2024-09-16T08:46:41.782 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.57262728Z level=info msg="Migration successfully executed" id="create test_data table" duration=542.73µs 2024-09-16T08:46:41.782 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.573008452Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-16T08:46:41.782 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.573522904Z level=info msg="Migration successfully executed" id="create dashboard_version table v1" duration=513.431µs 2024-09-16T08:46:41.782 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.573883999Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-16T08:46:41.783 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.574405743Z level=info msg="Migration successfully executed" id="add index dashboard_version.dashboard_id" duration=521.595µs 2024-09-16T08:46:41.783 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.574774247Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-16T08:46:41.783 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.575328933Z level=info msg="Migration successfully executed" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" duration=554.647µs 2024-09-16T08:46:41.783 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.577837563Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-16T08:46:41.783 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.577952283Z level=info msg="Migration successfully executed" id="Set dashboard version to 1 where 0" duration=114.375µs 2024-09-16T08:46:41.783 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.580642121Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-16T08:46:41.783 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.581008067Z level=info msg="Migration successfully executed" id="save existing dashboard data in dashboard_version table v1" duration=368.12µs 2024-09-16T08:46:41.783 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.583434788Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-16T08:46:41.783 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.583489513Z level=info msg="Migration successfully executed" id="alter dashboard_version.data to mediumtext v1" duration=54.927µs 2024-09-16T08:46:41.783 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.586272935Z level=info msg="Executing migration" id="create team table" 2024-09-16T08:46:41.783 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.586793255Z level=info msg="Migration successfully executed" id="create team table" duration=519.878µs 2024-09-16T08:46:41.783 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.589151835Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-16T08:46:41.783 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.589814169Z level=info msg="Migration successfully executed" id="add index team.org_id" duration=661.837µs 2024-09-16T08:46:41.784 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.591976964Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-16T08:46:41.784 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.592547667Z level=info msg="Migration successfully executed" id="add unique index team_org_id_name" duration=571.146µs 2024-09-16T08:46:41.784 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.594768379Z level=info msg="Executing migration" id="Add column uid in team" 2024-09-16T08:46:41.784 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.597299487Z level=info msg="Migration successfully executed" id="Add column uid in team" duration=2.530779ms 2024-09-16T08:46:41.784 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.597652139Z level=info msg="Executing migration" id="Update uid column values in team" 2024-09-16T08:46:41.784 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.597759311Z level=info msg="Migration successfully executed" id="Update uid column values in team" duration=107.993µs 2024-09-16T08:46:41.784 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.600614544Z level=info msg="Executing migration" id="Add unique index team_org_id_uid" 2024-09-16T08:46:41.784 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.60118873Z level=info msg="Migration successfully executed" id="Add unique index team_org_id_uid" duration=574.186µs 2024-09-16T08:46:41.785 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.603606999Z level=info msg="Executing migration" id="create team member table" 2024-09-16T08:46:41.785 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.604127122Z level=info msg="Migration successfully executed" id="create team member table" duration=519.87µs 2024-09-16T08:46:41.785 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.606355251Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-16T08:46:41.785 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.607051106Z level=info msg="Migration successfully executed" id="add index team_member.org_id" duration=698.216µs 2024-09-16T08:46:41.785 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.609194905Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-16T08:46:41.786 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.609750232Z level=info msg="Migration successfully executed" id="add unique index team_member_org_id_team_id_user_id" duration=554.722µs 2024-09-16T08:46:41.786 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.612218491Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-16T08:46:41.786 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.612865673Z level=info msg="Migration successfully executed" id="add index team_member.team_id" duration=649.298µs 2024-09-16T08:46:41.786 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.61493328Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-16T08:46:41.786 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.617670024Z level=info msg="Migration successfully executed" id="Add column email to team table" duration=2.736414ms 2024-09-16T08:46:41.786 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.618034034Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-16T08:46:41.786 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.620708965Z level=info msg="Migration successfully executed" id="Add column external to team_member table" duration=2.674697ms 2024-09-16T08:46:41.786 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.621069446Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-16T08:46:41.786 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.623625821Z level=info msg="Migration successfully executed" id="Add column permission to team_member table" duration=2.55617ms 2024-09-16T08:46:41.786 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.62415382Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-16T08:46:41.786 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.62482253Z level=info msg="Migration successfully executed" id="create dashboard acl table" duration=668.363µs 2024-09-16T08:46:41.786 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.627000966Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-16T08:46:41.786 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.627589364Z level=info msg="Migration successfully executed" id="add index dashboard_acl_dashboard_id" duration=588.32µs 2024-09-16T08:46:41.787 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.6279596Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-16T08:46:41.787 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.628583593Z level=info msg="Migration successfully executed" id="add unique index dashboard_acl_dashboard_id_user_id" duration=625.001µs 2024-09-16T08:46:41.787 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.630687728Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-16T08:46:41.787 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.631319927Z level=info msg="Migration successfully executed" id="add unique index dashboard_acl_dashboard_id_team_id" duration=631.989µs 2024-09-16T08:46:41.787 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.633328303Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-16T08:46:41.787 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.633887091Z level=info msg="Migration successfully executed" id="add index dashboard_acl_user_id" duration=558.613µs 2024-09-16T08:46:41.787 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.635969179Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-16T08:46:41.787 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.636522406Z level=info msg="Migration successfully executed" id="add index dashboard_acl_team_id" duration=553.277µs 2024-09-16T08:46:41.787 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.638674482Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-16T08:46:41.787 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.639286553Z level=info msg="Migration successfully executed" id="add index dashboard_acl_org_id_role" duration=611.669µs 2024-09-16T08:46:41.787 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.639637171Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-16T08:46:41.787 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.640223174Z level=info msg="Migration successfully executed" id="add index dashboard_permission" duration=585.804µs 2024-09-16T08:46:41.787 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.642411825Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-16T08:46:41.787 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.642806839Z level=info msg="Migration successfully executed" id="save default acl rules in dashboard_acl table" duration=397.328µs 2024-09-16T08:46:41.788 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.644892067Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-16T08:46:41.788 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.645026975Z level=info msg="Migration successfully executed" id="delete acl rules for deleted dashboards and folders" duration=134.878µs 2024-09-16T08:46:41.788 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.647540927Z level=info msg="Executing migration" id="create tag table" 2024-09-16T08:46:41.788 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.648040222Z level=info msg="Migration successfully executed" id="create tag table" duration=499.086µs 2024-09-16T08:46:41.788 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.648413862Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-16T08:46:41.788 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.648993203Z level=info msg="Migration successfully executed" id="add index tag.key_value" duration=577.358µs 2024-09-16T08:46:41.788 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.651183785Z level=info msg="Executing migration" id="create login attempt table" 2024-09-16T08:46:41.788 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.651695898Z level=info msg="Migration successfully executed" id="create login attempt table" duration=511.842µs 2024-09-16T08:46:41.788 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.653736118Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-16T08:46:41.788 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.654308424Z level=info msg="Migration successfully executed" id="add index login_attempt.username" duration=572.064µs 2024-09-16T08:46:41.788 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.65649363Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-16T08:46:41.788 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.657066758Z level=info msg="Migration successfully executed" id="drop index IDX_login_attempt_username - v1" duration=573.049µs 2024-09-16T08:46:41.789 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.657410199Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-16T08:46:41.789 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.665268183Z level=info msg="Migration successfully executed" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" duration=7.859201ms 2024-09-16T08:46:41.789 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.665630658Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-16T08:46:41.789 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.666104935Z level=info msg="Migration successfully executed" id="create login_attempt v2" duration=474.2µs 2024-09-16T08:46:41.789 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.668339932Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-16T08:46:41.789 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.669023266Z level=info msg="Migration successfully executed" id="create index IDX_login_attempt_username - v2" duration=685.465µs 2024-09-16T08:46:41.789 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.670839221Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-16T08:46:41.789 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.671037307Z level=info msg="Migration successfully executed" id="copy login_attempt v1 to v2" duration=197.89µs 2024-09-16T08:46:41.789 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.673802251Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-16T08:46:41.789 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.67425642Z level=info msg="Migration successfully executed" id="drop login_attempt_tmp_qwerty" duration=456.531µs 2024-09-16T08:46:41.789 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.67461615Z level=info msg="Executing migration" id="create user auth table" 2024-09-16T08:46:41.789 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.67510575Z level=info msg="Migration successfully executed" id="create user auth table" duration=489.421µs 2024-09-16T08:46:41.789 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.675468315Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-16T08:46:41.790 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.676074655Z level=info msg="Migration successfully executed" id="create index IDX_user_auth_auth_module_auth_id - v1" duration=605.46µs 2024-09-16T08:46:41.790 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.676433346Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-16T08:46:41.790 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.676474291Z level=info msg="Migration successfully executed" id="alter user_auth.auth_id to length 190" duration=41.184µs 2024-09-16T08:46:41.790 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.676892202Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-16T08:46:41.790 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.679848413Z level=info msg="Migration successfully executed" id="Add OAuth access token to user_auth" duration=2.95579ms 2024-09-16T08:46:41.790 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.680180277Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-16T08:46:41.790 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.683029191Z level=info msg="Migration successfully executed" id="Add OAuth refresh token to user_auth" duration=2.848131ms 2024-09-16T08:46:41.790 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.683371979Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-16T08:46:41.790 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.686253631Z level=info msg="Migration successfully executed" id="Add OAuth token type to user_auth" duration=2.881391ms 2024-09-16T08:46:41.790 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.686601206Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-16T08:46:41.790 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.689468451Z level=info msg="Migration successfully executed" id="Add OAuth expiry to user_auth" duration=2.866815ms 2024-09-16T08:46:41.790 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.689825492Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-16T08:46:41.790 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.690402467Z level=info msg="Migration successfully executed" id="Add index to user_id column in user_auth" duration=576.881µs 2024-09-16T08:46:41.791 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.690751659Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-16T08:46:41.791 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.693614793Z level=info msg="Migration successfully executed" id="Add OAuth ID token to user_auth" duration=2.862993ms 2024-09-16T08:46:41.791 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.69396489Z level=info msg="Executing migration" id="create server_lock table" 2024-09-16T08:46:41.791 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.694465927Z level=info msg="Migration successfully executed" id="create server_lock table" duration=500.708µs 2024-09-16T08:46:41.791 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.694826122Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-16T08:46:41.791 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.695376978Z level=info msg="Migration successfully executed" id="add index server_lock.operation_uid" duration=550.769µs 2024-09-16T08:46:41.791 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.69572575Z level=info msg="Executing migration" id="create user auth token table" 2024-09-16T08:46:41.791 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.696277381Z level=info msg="Migration successfully executed" id="create user auth token table" duration=551.525µs 2024-09-16T08:46:41.791 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.69663038Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-16T08:46:41.791 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.697194651Z level=info msg="Migration successfully executed" id="add unique index user_auth_token.auth_token" duration=564.16µs 2024-09-16T08:46:41.791 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.697547152Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-16T08:46:41.791 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.698122745Z level=info msg="Migration successfully executed" id="add unique index user_auth_token.prev_auth_token" duration=575.57µs 2024-09-16T08:46:41.791 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.698475923Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-16T08:46:41.792 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.699114742Z level=info msg="Migration successfully executed" id="add index user_auth_token.user_id" duration=637.59µs 2024-09-16T08:46:41.792 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.699469953Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-16T08:46:41.792 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.702488442Z level=info msg="Migration successfully executed" id="Add revoked_at to the user auth token" duration=3.018437ms 2024-09-16T08:46:41.792 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.702837174Z level=info msg="Executing migration" id="add index user_auth_token.revoked_at" 2024-09-16T08:46:41.792 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.703407097Z level=info msg="Migration successfully executed" id="add index user_auth_token.revoked_at" duration=569.863µs 2024-09-16T08:46:41.792 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.703754349Z level=info msg="Executing migration" id="create cache_data table" 2024-09-16T08:46:41.792 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.704304889Z level=info msg="Migration successfully executed" id="create cache_data table" duration=550.458µs 2024-09-16T08:46:41.792 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.704658206Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-16T08:46:41.792 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.705229656Z level=info msg="Migration successfully executed" id="add unique index cache_data.cache_key" duration=571.309µs 2024-09-16T08:46:41.792 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.70557594Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-16T08:46:41.792 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.706150409Z level=info msg="Migration successfully executed" id="create short_url table v1" duration=574.417µs 2024-09-16T08:46:41.792 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.706547537Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-16T08:46:41.793 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.707232965Z level=info msg="Migration successfully executed" id="add index short_url.org_id-uid" duration=684.631µs 2024-09-16T08:46:41.793 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.70759637Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-16T08:46:41.793 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.707635466Z level=info msg="Migration successfully executed" id="alter table short_url alter column created_by type to bigint" duration=39.361µs 2024-09-16T08:46:41.793 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.70806433Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-16T08:46:41.793 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.708150256Z level=info msg="Migration successfully executed" id="delete alert_definition table" duration=86.241µs 2024-09-16T08:46:41.793 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.708529317Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-16T08:46:41.793 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.709103209Z level=info msg="Migration successfully executed" id="recreate alert_definition table" duration=573.663µs 2024-09-16T08:46:41.793 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.709467553Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-16T08:46:41.793 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.710108065Z level=info msg="Migration successfully executed" id="add index in alert_definition on org_id and title columns" duration=640.661µs 2024-09-16T08:46:41.793 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.71048033Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-16T08:46:41.793 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.711115506Z level=info msg="Migration successfully executed" id="add index in alert_definition on org_id and uid columns" duration=634.785µs 2024-09-16T08:46:41.793 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.711459216Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-16T08:46:41.794 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.711501671Z level=info msg="Migration successfully executed" id="alter alert_definition table data column to mediumtext in mysql" duration=43.147µs 2024-09-16T08:46:41.794 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.711896171Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-16T08:46:41.794 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.712482716Z level=info msg="Migration successfully executed" id="drop index in alert_definition on org_id and title columns" duration=586.85µs 2024-09-16T08:46:41.794 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.712839871Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-16T08:46:41.794 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.713426827Z level=info msg="Migration successfully executed" id="drop index in alert_definition on org_id and uid columns" duration=587.304µs 2024-09-16T08:46:41.794 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.713794032Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-16T08:46:41.794 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.714416693Z level=info msg="Migration successfully executed" id="add unique index in alert_definition on org_id and title columns" duration=621.9µs 2024-09-16T08:46:41.794 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.714784951Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-16T08:46:41.794 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.715408042Z level=info msg="Migration successfully executed" id="add unique index in alert_definition on org_id and uid columns" duration=623.357µs 2024-09-16T08:46:41.794 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.715756074Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-16T08:46:41.794 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.719143117Z level=info msg="Migration successfully executed" id="Add column paused in alert_definition" duration=3.387194ms 2024-09-16T08:46:41.794 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.719503104Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-16T08:46:41.794 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.720162551Z level=info msg="Migration successfully executed" id="drop alert_definition table" duration=659.284µs 2024-09-16T08:46:41.795 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.720529235Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-16T08:46:41.795 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.720596141Z level=info msg="Migration successfully executed" id="delete alert_definition_version table" duration=65.678µs 2024-09-16T08:46:41.795 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.721018465Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-16T08:46:41.795 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.721613932Z level=info msg="Migration successfully executed" id="recreate alert_definition_version table" duration=595.2µs 2024-09-16T08:46:41.795 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.721966724Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-16T08:46:41.795 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.72259265Z level=info msg="Migration successfully executed" id="add index in alert_definition_version table on alert_definition_id and version columns" duration=626.02µs 2024-09-16T08:46:41.796 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.722944448Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-16T08:46:41.796 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.723570609Z level=info msg="Migration successfully executed" id="add index in alert_definition_version table on alert_definition_uid and version columns" duration=626.281µs 2024-09-16T08:46:41.796 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.723902028Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-16T08:46:41.796 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.723940912Z level=info msg="Migration successfully executed" id="alter alert_definition_version table data column to mediumtext in mysql" duration=39.201µs 2024-09-16T08:46:41.796 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.724342436Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-16T08:46:41.796 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.72501282Z level=info msg="Migration successfully executed" id="drop alert_definition_version table" duration=670.597µs 2024-09-16T08:46:41.796 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.725399769Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-16T08:46:41.796 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.726011516Z level=info msg="Migration successfully executed" id="create alert_instance table" duration=611.319µs 2024-09-16T08:46:41.796 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.726391067Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-16T08:46:41.796 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.727034154Z level=info msg="Migration successfully executed" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" duration=627.036µs 2024-09-16T08:46:41.796 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.727391106Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-16T08:46:41.797 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.728017602Z level=info msg="Migration successfully executed" id="add index in alert_instance table on def_org_id, current_state columns" duration=626.117µs 2024-09-16T08:46:41.797 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.728352177Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-16T08:46:41.797 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.73163809Z level=info msg="Migration successfully executed" id="add column current_state_end to alert_instance" duration=3.286246ms 2024-09-16T08:46:41.797 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.731999408Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-16T08:46:41.797 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.732581238Z level=info msg="Migration successfully executed" id="remove index def_org_id, def_uid, current_state on alert_instance" duration=581.78µs 2024-09-16T08:46:41.797 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.732949479Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-16T08:46:41.797 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.733555817Z level=info msg="Migration successfully executed" id="remove index def_org_id, current_state on alert_instance" duration=606.234µs 2024-09-16T08:46:41.797 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.733920188Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-16T08:46:41.797 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.751301585Z level=info msg="Migration successfully executed" id="rename def_org_id to rule_org_id in alert_instance" duration=17.381129ms 2024-09-16T08:46:41.797 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.75169599Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-16T08:46:41.797 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.767907148Z level=info msg="Migration successfully executed" id="rename def_uid to rule_uid in alert_instance" duration=16.212512ms 2024-09-16T08:46:41.797 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.768269413Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-16T08:46:41.797 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.769012405Z level=info msg="Migration successfully executed" id="add index rule_org_id, rule_uid, current_state on alert_instance" duration=743.54µs 2024-09-16T08:46:41.798 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.769370759Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-16T08:46:41.798 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.769966103Z level=info msg="Migration successfully executed" id="add index rule_org_id, current_state on alert_instance" duration=595.108µs 2024-09-16T08:46:41.798 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.770327091Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-16T08:46:41.798 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.773537097Z level=info msg="Migration successfully executed" id="add current_reason column related to current_state" duration=3.209814ms 2024-09-16T08:46:41.798 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.773902986Z level=info msg="Executing migration" id="add result_fingerprint column to alert_instance" 2024-09-16T08:46:41.798 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.777135349Z level=info msg="Migration successfully executed" id="add result_fingerprint column to alert_instance" duration=3.232125ms 2024-09-16T08:46:41.798 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.777520918Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-16T08:46:41.798 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.77816028Z level=info msg="Migration successfully executed" id="create alert_rule table" duration=639.246µs 2024-09-16T08:46:41.798 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.77861589Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-16T08:46:41.798 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.779584739Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id and title columns" duration=968.825µs 2024-09-16T08:46:41.798 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.780054592Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-16T08:46:41.798 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.780967262Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id and uid columns" duration=912.309µs 2024-09-16T08:46:41.798 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.781416744Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-16T08:46:41.799 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.782413904Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" duration=996.76µs 2024-09-16T08:46:41.799 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.782866735Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-16T08:46:41.799 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.782919844Z level=info msg="Migration successfully executed" id="alter alert_rule table data column to mediumtext in mysql" duration=53.512µs 2024-09-16T08:46:41.799 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.783735912Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-16T08:46:42.030 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.789138855Z level=info msg="Migration successfully executed" id="add column for to alert_rule" duration=5.403266ms 2024-09-16T08:46:42.031 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.789484986Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-16T08:46:42.031 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.792815046Z level=info msg="Migration successfully executed" id="add column annotations to alert_rule" duration=3.329783ms 2024-09-16T08:46:42.031 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.793177093Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-16T08:46:42.031 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.796516372Z level=info msg="Migration successfully executed" id="add column labels to alert_rule" duration=3.338953ms 2024-09-16T08:46:42.031 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.79687135Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-16T08:46:42.031 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.797441209Z level=info msg="Migration successfully executed" id="remove unique index from alert_rule on org_id, title columns" duration=572.196µs 2024-09-16T08:46:42.031 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.797831135Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-16T08:46:42.031 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.798453964Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, namespase_uid and title columns" duration=622.403µs 2024-09-16T08:46:42.031 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.798801955Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-16T08:46:42.031 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.802142089Z level=info msg="Migration successfully executed" id="add dashboard_uid column to alert_rule" duration=3.33973ms 2024-09-16T08:46:42.031 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.802491201Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-16T08:46:42.031 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.805830988Z level=info msg="Migration successfully executed" id="add panel_id column to alert_rule" duration=3.339557ms 2024-09-16T08:46:42.032 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.8061816Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-16T08:46:42.032 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.806816788Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" duration=634.685µs 2024-09-16T08:46:42.032 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.807176557Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-16T08:46:42.032 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.810529892Z level=info msg="Migration successfully executed" id="add rule_group_idx column to alert_rule" duration=3.356462ms 2024-09-16T08:46:42.032 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.810893608Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-16T08:46:42.032 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.81426971Z level=info msg="Migration successfully executed" id="add is_paused column to alert_rule table" duration=3.376365ms 2024-09-16T08:46:42.033 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.814633778Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-16T08:46:42.033 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.814672331Z level=info msg="Migration successfully executed" id="fix is_paused column for alert_rule table" duration=38.9µs 2024-09-16T08:46:42.033 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.815102104Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-16T08:46:42.033 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.815866796Z level=info msg="Migration successfully executed" id="create alert_rule_version table" duration=764.406µs 2024-09-16T08:46:42.033 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.817832458Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-16T08:46:42.033 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.818507571Z level=info msg="Migration successfully executed" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" duration=674.844µs 2024-09-16T08:46:42.033 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.818871231Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-16T08:46:42.033 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.819584353Z level=info msg="Migration successfully executed" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" duration=713.838µs 2024-09-16T08:46:42.033 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.821675486Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-16T08:46:42.033 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.821735861Z level=info msg="Migration successfully executed" id="alter alert_rule_version table data column to mediumtext in mysql" duration=60.509µs 2024-09-16T08:46:42.033 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.822149855Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-16T08:46:42.033 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.825717603Z level=info msg="Migration successfully executed" id="add column for to alert_rule_version" duration=3.566359ms 2024-09-16T08:46:42.034 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.82609155Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-16T08:46:42.034 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.829715743Z level=info msg="Migration successfully executed" id="add column annotations to alert_rule_version" duration=3.624034ms 2024-09-16T08:46:42.034 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.830089681Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-16T08:46:42.034 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.833578337Z level=info msg="Migration successfully executed" id="add column labels to alert_rule_version" duration=3.488516ms 2024-09-16T08:46:42.034 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.833950422Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-16T08:46:42.034 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.837455968Z level=info msg="Migration successfully executed" id="add rule_group_idx column to alert_rule_version" duration=3.505849ms 2024-09-16T08:46:42.034 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.837822992Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-16T08:46:42.034 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.841344062Z level=info msg="Migration successfully executed" id="add is_paused column to alert_rule_versions table" duration=3.520589ms 2024-09-16T08:46:42.034 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.841700025Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-16T08:46:42.034 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.841738735Z level=info msg="Migration successfully executed" id="fix is_paused column for alert_rule_version table" duration=39.03µs 2024-09-16T08:46:42.034 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.842495666Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-16T08:46:42.034 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.84300045Z level=info msg="Migration successfully executed" id=create_alert_configuration_table duration=502.83µs 2024-09-16T08:46:42.034 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.84341743Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-16T08:46:42.035 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.847001711Z level=info msg="Migration successfully executed" id="Add column default in alert_configuration" duration=3.584036ms 2024-09-16T08:46:42.035 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.847386195Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-16T08:46:42.035 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.847442866Z level=info msg="Migration successfully executed" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" duration=58.327µs 2024-09-16T08:46:42.035 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.848379854Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-16T08:46:42.035 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.852842114Z level=info msg="Migration successfully executed" id="add column org_id in alert_configuration" duration=4.463011ms 2024-09-16T08:46:42.035 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.853229411Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-16T08:46:42.035 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.853951896Z level=info msg="Migration successfully executed" id="add index in alert_configuration table on org_id column" duration=722.407µs 2024-09-16T08:46:42.035 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.854332674Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-16T08:46:42.035 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.858182777Z level=info msg="Migration successfully executed" id="add configuration_hash column to alert_configuration" duration=3.849828ms 2024-09-16T08:46:42.035 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.858578587Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-16T08:46:42.036 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.859107808Z level=info msg="Migration successfully executed" id=create_ngalert_configuration_table duration=529.089µs 2024-09-16T08:46:42.036 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.861361808Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-16T08:46:42.036 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.861981397Z level=info msg="Migration successfully executed" id="add index in ngalert_configuration on org_id column" duration=617.694µs 2024-09-16T08:46:42.036 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.864287402Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-16T08:46:42.036 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.868038884Z level=info msg="Migration successfully executed" id="add column send_alerts_to in ngalert_configuration" duration=3.753325ms 2024-09-16T08:46:42.036 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.868457315Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-16T08:46:42.036 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.868971823Z level=info msg="Migration successfully executed" id="create provenance_type table" duration=514.337µs 2024-09-16T08:46:42.036 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.869402939Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-16T08:46:42.036 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.870056574Z level=info msg="Migration successfully executed" id="add index to uniquify (record_key, record_type, org_id) columns" duration=653.37µs 2024-09-16T08:46:42.036 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.872140258Z level=info msg="Executing migration" id="create alert_image table" 2024-09-16T08:46:42.036 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.872713707Z level=info msg="Migration successfully executed" id="create alert_image table" duration=573.353µs 2024-09-16T08:46:42.036 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.875047026Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-16T08:46:42.036 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.875793836Z level=info msg="Migration successfully executed" id="add unique index on token to alert_image table" duration=755.044µs 2024-09-16T08:46:42.037 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.877724801Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-16T08:46:42.037 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.877855942Z level=info msg="Migration successfully executed" id="support longer URLs in alert_image table" duration=131.728µs 2024-09-16T08:46:42.038 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.878339744Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-16T08:46:42.038 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.878956074Z level=info msg="Migration successfully executed" id=create_alert_configuration_history_table duration=616.074µs 2024-09-16T08:46:42.038 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.881232587Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-16T08:46:42.038 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.881888772Z level=info msg="Migration successfully executed" id="drop non-unique orgID index on alert_configuration" duration=655.982µs 2024-09-16T08:46:42.038 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.88403075Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-16T08:46:42.038 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.884263523Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-16T08:46:42.038 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.886545741Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-16T08:46:42.038 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.886861283Z level=info msg="Migration successfully executed" id="extract alertmanager configuration history to separate table" duration=314.265µs 2024-09-16T08:46:42.038 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.887231122Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-16T08:46:42.038 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.887855838Z level=info msg="Migration successfully executed" id="add unique index on orgID to alert_configuration" duration=624.958µs 2024-09-16T08:46:42.038 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.89005736Z level=info msg="Executing migration" id="add last_applied column to alert_configuration_history" 2024-09-16T08:46:42.038 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.893909147Z level=info msg="Migration successfully executed" id="add last_applied column to alert_configuration_history" duration=3.854349ms 2024-09-16T08:46:42.039 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.89430292Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-16T08:46:42.039 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.895020868Z level=info msg="Migration successfully executed" id="create library_element table v1" duration=718.682µs 2024-09-16T08:46:42.039 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.896988616Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-16T08:46:42.039 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.897653832Z level=info msg="Migration successfully executed" id="add index library_element org_id-folder_id-name-kind" duration=664.955µs 2024-09-16T08:46:42.039 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.898033502Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-16T08:46:42.039 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.898569786Z level=info msg="Migration successfully executed" id="create library_element_connection table v1" duration=536.126µs 2024-09-16T08:46:42.039 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.900513544Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-16T08:46:42.039 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.901213083Z level=info msg="Migration successfully executed" id="add index library_element_connection element_id-kind-connection_id" duration=699.305µs 2024-09-16T08:46:42.039 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.903025142Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-16T08:46:42.039 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.903775286Z level=info msg="Migration successfully executed" id="add unique index library_element org_id_uid" duration=751.028µs 2024-09-16T08:46:42.039 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.905501515Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-16T08:46:42.039 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.905533851Z level=info msg="Migration successfully executed" id="increase max description length to 2048" duration=32.851µs 2024-09-16T08:46:42.039 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.907935471Z level=info msg="Executing migration" id="alter library_element model to mediumtext" 2024-09-16T08:46:42.040 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.907977958Z level=info msg="Migration successfully executed" id="alter library_element model to mediumtext" duration=42.761µs 2024-09-16T08:46:42.040 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.910371011Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-16T08:46:42.040 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.910561516Z level=info msg="Migration successfully executed" id="clone move dashboard alerts to unified alerting" duration=190.325µs 2024-09-16T08:46:42.040 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.912929681Z level=info msg="Executing migration" id="create data_keys table" 2024-09-16T08:46:42.040 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.913561952Z level=info msg="Migration successfully executed" id="create data_keys table" duration=633.079µs 2024-09-16T08:46:42.040 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.915600534Z level=info msg="Executing migration" id="create secrets table" 2024-09-16T08:46:42.040 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.916158618Z level=info msg="Migration successfully executed" id="create secrets table" duration=557.661µs 2024-09-16T08:46:42.040 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.916542435Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-16T08:46:42.040 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.93790403Z level=info msg="Migration successfully executed" id="rename data_keys name column to id" duration=21.359784ms 2024-09-16T08:46:42.040 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.938268393Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-16T08:46:42.040 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.942387737Z level=info msg="Migration successfully executed" id="add name column into data_keys" duration=4.11904ms 2024-09-16T08:46:42.041 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.942742712Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-16T08:46:42.041 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.94285586Z level=info msg="Migration successfully executed" id="copy data_keys id column values into name" duration=111.589µs 2024-09-16T08:46:42.041 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.945265893Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-16T08:46:42.041 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.96600045Z level=info msg="Migration successfully executed" id="rename data_keys name column to label" duration=20.734072ms 2024-09-16T08:46:42.041 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.966363941Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-16T08:46:42.041 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.987477192Z level=info msg="Migration successfully executed" id="rename data_keys id column back to name" duration=21.114258ms 2024-09-16T08:46:42.041 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.987864508Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-16T08:46:42.041 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.988425039Z level=info msg="Migration successfully executed" id="create kv_store table v1" duration=560.411µs 2024-09-16T08:46:42.041 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.990415931Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-16T08:46:42.041 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.991135907Z level=info msg="Migration successfully executed" id="add index kv_store.org_id-namespace-key" duration=719.917µs 2024-09-16T08:46:42.041 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.992925667Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-16T08:46:42.041 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.993064672Z level=info msg="Migration successfully executed" id="update dashboard_uid and panel_id from existing annotations" duration=139.274µs 2024-09-16T08:46:42.041 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.993464647Z level=info msg="Executing migration" id="create permission table" 2024-09-16T08:46:42.042 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.994076242Z level=info msg="Migration successfully executed" id="create permission table" duration=611.397µs 2024-09-16T08:46:42.042 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.996094552Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-16T08:46:42.042 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.996725003Z level=info msg="Migration successfully executed" id="add unique index permission.role_id" duration=630.139µs 2024-09-16T08:46:42.042 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.998710123Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-16T08:46:42.042 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.999442607Z level=info msg="Migration successfully executed" id="add unique index role_id_action_scope" duration=734.164µs 2024-09-16T08:46:42.042 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:41 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:41.999818819Z level=info msg="Executing migration" id="create role table" 2024-09-16T08:46:42.042 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.000393888Z level=info msg="Migration successfully executed" id="create role table" duration=574.537µs 2024-09-16T08:46:42.042 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.000746997Z level=info msg="Executing migration" id="add column display_name" 2024-09-16T08:46:42.042 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.00520911Z level=info msg="Migration successfully executed" id="add column display_name" duration=4.462018ms 2024-09-16T08:46:42.042 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.00558287Z level=info msg="Executing migration" id="add column group_name" 2024-09-16T08:46:42.042 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.009746585Z level=info msg="Migration successfully executed" id="add column group_name" duration=4.163452ms 2024-09-16T08:46:42.042 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.010125675Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-16T08:46:42.042 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.010756424Z level=info msg="Migration successfully executed" id="add index role.org_id" duration=630.673µs 2024-09-16T08:46:42.043 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.011140505Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-16T08:46:42.043 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.011809491Z level=info msg="Migration successfully executed" id="add unique index role_org_id_name" duration=670.087µs 2024-09-16T08:46:42.043 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.013942188Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-16T08:46:42.043 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.014585609Z level=info msg="Migration successfully executed" id="add index role_org_id_uid" duration=645.295µs 2024-09-16T08:46:42.043 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.014937492Z level=info msg="Executing migration" id="create team role table" 2024-09-16T08:46:42.043 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.01546469Z level=info msg="Migration successfully executed" id="create team role table" duration=527.014µs 2024-09-16T08:46:42.043 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.015810363Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-16T08:46:42.043 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.016480066Z level=info msg="Migration successfully executed" id="add index team_role.org_id" duration=669.454µs 2024-09-16T08:46:42.043 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.016842801Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-16T08:46:42.043 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.017527265Z level=info msg="Migration successfully executed" id="add unique index team_role_org_id_team_id_role_id" duration=684.151µs 2024-09-16T08:46:42.043 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.017881686Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-16T08:46:42.043 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.018494493Z level=info msg="Migration successfully executed" id="add index team_role.team_id" duration=612.606µs 2024-09-16T08:46:42.043 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.018835148Z level=info msg="Executing migration" id="create user role table" 2024-09-16T08:46:42.044 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.019366499Z level=info msg="Migration successfully executed" id="create user role table" duration=531.26µs 2024-09-16T08:46:42.044 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.019732883Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-16T08:46:42.044 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.020377687Z level=info msg="Migration successfully executed" id="add index user_role.org_id" duration=644.684µs 2024-09-16T08:46:42.044 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.020723672Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-16T08:46:42.044 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.021373192Z level=info msg="Migration successfully executed" id="add unique index user_role_org_id_user_id_role_id" duration=649.253µs 2024-09-16T08:46:42.044 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.021719357Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-16T08:46:42.044 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.022364286Z level=info msg="Migration successfully executed" id="add index user_role.user_id" duration=644.869µs 2024-09-16T08:46:42.044 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.022756563Z level=info msg="Executing migration" id="create builtin role table" 2024-09-16T08:46:42.044 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.023294998Z level=info msg="Migration successfully executed" id="create builtin role table" duration=538.411µs 2024-09-16T08:46:42.044 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.023643494Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-16T08:46:42.044 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.024298113Z level=info msg="Migration successfully executed" id="add index builtin_role.role_id" duration=654.288µs 2024-09-16T08:46:42.045 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.024651882Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-16T08:46:42.045 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.025312518Z level=info msg="Migration successfully executed" id="add index builtin_role.name" duration=660.447µs 2024-09-16T08:46:42.045 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.025663252Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-16T08:46:42.045 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.030290695Z level=info msg="Migration successfully executed" id="Add column org_id to builtin_role table" duration=4.628626ms 2024-09-16T08:46:42.045 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.030705024Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-16T08:46:42.046 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.031671554Z level=info msg="Migration successfully executed" id="add index builtin_role.org_id" duration=966.141µs 2024-09-16T08:46:42.046 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.032120167Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-16T08:46:42.046 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.03310934Z level=info msg="Migration successfully executed" id="add unique index builtin_role_org_id_role_id_role" duration=988.879µs 2024-09-16T08:46:42.046 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.033546708Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-16T08:46:42.046 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.034513026Z level=info msg="Migration successfully executed" id="Remove unique index role_org_id_uid" duration=967.585µs 2024-09-16T08:46:42.046 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.034943029Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-16T08:46:42.046 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.03590841Z level=info msg="Migration successfully executed" id="add unique index role.uid" duration=965.286µs 2024-09-16T08:46:42.046 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.036337103Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-16T08:46:42.046 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.03705952Z level=info msg="Migration successfully executed" id="create seed assignment table" duration=722.322µs 2024-09-16T08:46:42.046 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.037496928Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-16T08:46:42.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:42 smithi160 ceph-mon[30137]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:42.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:42 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:42.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:42 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:42.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:42 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:42.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:42 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:42.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:42 smithi160 ceph-mon[30137]: Deploying daemon node-exporter.a on smithi160 2024-09-16T08:46:42.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:42 smithi160 ceph-mon[40028]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:42.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:42 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:42.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:42 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:42.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:42 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:42.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:42 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:42.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:42 smithi160 ceph-mon[40028]: Deploying daemon node-exporter.a on smithi160 2024-09-16T08:46:42.296 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-mon[33465]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:42.296 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:42.296 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:42.296 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:42.296 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:42.297 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-mon[33465]: Deploying daemon node-exporter.a on smithi160 2024-09-16T08:46:42.297 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.038290252Z level=info msg="Migration successfully executed" id="add unique index builtin_role_role_name" duration=793.633µs 2024-09-16T08:46:42.297 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.038635764Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-16T08:46:42.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.043292871Z level=info msg="Migration successfully executed" id="add column hidden to role table" duration=4.655578ms 2024-09-16T08:46:42.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.043622255Z level=info msg="Executing migration" id="permission kind migration" 2024-09-16T08:46:42.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.048170408Z level=info msg="Migration successfully executed" id="permission kind migration" duration=4.546074ms 2024-09-16T08:46:42.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.048524065Z level=info msg="Executing migration" id="permission attribute migration" 2024-09-16T08:46:42.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.052964021Z level=info msg="Migration successfully executed" id="permission attribute migration" duration=4.439631ms 2024-09-16T08:46:42.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.053292728Z level=info msg="Executing migration" id="permission identifier migration" 2024-09-16T08:46:42.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.057731748Z level=info msg="Migration successfully executed" id="permission identifier migration" duration=4.438941ms 2024-09-16T08:46:42.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.058087179Z level=info msg="Executing migration" id="add permission identifier index" 2024-09-16T08:46:42.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.058740187Z level=info msg="Migration successfully executed" id="add permission identifier index" duration=652.841µs 2024-09-16T08:46:42.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.059091691Z level=info msg="Executing migration" id="add permission action scope role_id index" 2024-09-16T08:46:42.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.05980191Z level=info msg="Migration successfully executed" id="add permission action scope role_id index" duration=709.87µs 2024-09-16T08:46:42.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.060139135Z level=info msg="Executing migration" id="remove permission role_id action scope index" 2024-09-16T08:46:42.298 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.060760197Z level=info msg="Migration successfully executed" id="remove permission role_id action scope index" duration=621.024µs 2024-09-16T08:46:42.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.06109948Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-16T08:46:42.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.061652268Z level=info msg="Migration successfully executed" id="create query_history table v1" duration=552.663µs 2024-09-16T08:46:42.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.061991421Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-16T08:46:42.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.062630308Z level=info msg="Migration successfully executed" id="add index query_history.org_id-created_by-datasource_uid" duration=638.527µs 2024-09-16T08:46:42.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.062968522Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-16T08:46:42.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.063008106Z level=info msg="Migration successfully executed" id="alter table query_history alter column created_by type to bigint" duration=39.862µs 2024-09-16T08:46:42.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.063693881Z level=info msg="Executing migration" id="rbac disabled migrator" 2024-09-16T08:46:42.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.06372313Z level=info msg="Migration successfully executed" id="rbac disabled migrator" duration=30.027µs 2024-09-16T08:46:42.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.066619198Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-16T08:46:42.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.0669655Z level=info msg="Migration successfully executed" id="teams permissions migration" duration=350.987µs 2024-09-16T08:46:42.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.06932571Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-16T08:46:42.299 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.069665545Z level=info msg="Migration successfully executed" id="dashboard permissions" duration=340.338µs 2024-09-16T08:46:42.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.070059075Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-16T08:46:42.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.070462955Z level=info msg="Migration successfully executed" id="dashboard permissions uid scopes" duration=404.925µs 2024-09-16T08:46:42.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.073083863Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-16T08:46:42.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.073248531Z level=info msg="Migration successfully executed" id="drop managed folder create actions" duration=164.894µs 2024-09-16T08:46:42.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.075690663Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-16T08:46:42.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.076005506Z level=info msg="Migration successfully executed" id="alerting notification permissions" duration=314.594µs 2024-09-16T08:46:42.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.078529692Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-16T08:46:42.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.079080784Z level=info msg="Migration successfully executed" id="create query_history_star table v1" duration=550.886µs 2024-09-16T08:46:42.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.079438591Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-16T08:46:42.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.080110098Z level=info msg="Migration successfully executed" id="add index query_history.user_id-query_uid" duration=671.354µs 2024-09-16T08:46:42.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.082408743Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-16T08:46:42.300 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.087122175Z level=info msg="Migration successfully executed" id="add column org_id in query_history_star" duration=4.715524ms 2024-09-16T08:46:42.301 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.087465181Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-16T08:46:42.301 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.087503286Z level=info msg="Migration successfully executed" id="alter table query_history_star_mig column user_id type to bigint" duration=38.46µs 2024-09-16T08:46:42.301 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.090123913Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-16T08:46:42.301 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.090823852Z level=info msg="Migration successfully executed" id="create correlation table v1" duration=699.601µs 2024-09-16T08:46:42.301 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.091184356Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-16T08:46:42.301 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.091827839Z level=info msg="Migration successfully executed" id="add index correlations.uid" duration=641.676µs 2024-09-16T08:46:42.301 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.093916571Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-16T08:46:42.301 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.094567175Z level=info msg="Migration successfully executed" id="add index correlations.source_uid" duration=650.509µs 2024-09-16T08:46:42.301 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.096664433Z level=info msg="Executing migration" id="add correlation config column" 2024-09-16T08:46:42.301 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.10149832Z level=info msg="Migration successfully executed" id="add correlation config column" duration=4.833449ms 2024-09-16T08:46:42.301 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.101857877Z level=info msg="Executing migration" id="drop index IDX_correlation_uid - v1" 2024-09-16T08:46:42.301 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.10249758Z level=info msg="Migration successfully executed" id="drop index IDX_correlation_uid - v1" duration=639.613µs 2024-09-16T08:46:42.301 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.102840305Z level=info msg="Executing migration" id="drop index IDX_correlation_source_uid - v1" 2024-09-16T08:46:42.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.10347958Z level=info msg="Migration successfully executed" id="drop index IDX_correlation_source_uid - v1" duration=639.178µs 2024-09-16T08:46:42.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.105600041Z level=info msg="Executing migration" id="Rename table correlation to correlation_tmp_qwerty - v1" 2024-09-16T08:46:42.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.119124782Z level=info msg="Migration successfully executed" id="Rename table correlation to correlation_tmp_qwerty - v1" duration=13.525193ms 2024-09-16T08:46:42.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.119494372Z level=info msg="Executing migration" id="create correlation v2" 2024-09-16T08:46:42.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.120219815Z level=info msg="Migration successfully executed" id="create correlation v2" duration=725.042µs 2024-09-16T08:46:42.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.120589903Z level=info msg="Executing migration" id="create index IDX_correlation_uid - v2" 2024-09-16T08:46:42.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.121266463Z level=info msg="Migration successfully executed" id="create index IDX_correlation_uid - v2" duration=677.856µs 2024-09-16T08:46:42.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.121590527Z level=info msg="Executing migration" id="create index IDX_correlation_source_uid - v2" 2024-09-16T08:46:42.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.122293108Z level=info msg="Migration successfully executed" id="create index IDX_correlation_source_uid - v2" duration=702.363µs 2024-09-16T08:46:42.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.12262166Z level=info msg="Executing migration" id="create index IDX_correlation_org_id - v2" 2024-09-16T08:46:42.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.123279109Z level=info msg="Migration successfully executed" id="create index IDX_correlation_org_id - v2" duration=657.341µs 2024-09-16T08:46:42.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.123630302Z level=info msg="Executing migration" id="copy correlation v1 to v2" 2024-09-16T08:46:42.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.123797288Z level=info msg="Migration successfully executed" id="copy correlation v1 to v2" duration=166.974µs 2024-09-16T08:46:42.302 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.124391637Z level=info msg="Executing migration" id="drop correlation_tmp_qwerty" 2024-09-16T08:46:42.303 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.124926988Z level=info msg="Migration successfully executed" id="drop correlation_tmp_qwerty" duration=535.106µs 2024-09-16T08:46:42.303 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.125265359Z level=info msg="Executing migration" id="add provisioning column" 2024-09-16T08:46:42.303 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.130026213Z level=info msg="Migration successfully executed" id="add provisioning column" duration=4.760594ms 2024-09-16T08:46:42.303 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.130406741Z level=info msg="Executing migration" id="create entity_events table" 2024-09-16T08:46:42.303 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.130923769Z level=info msg="Migration successfully executed" id="create entity_events table" duration=516.894µs 2024-09-16T08:46:42.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.131269134Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-16T08:46:42.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.131954538Z level=info msg="Migration successfully executed" id="create dashboard public config v1" duration=685.105µs 2024-09-16T08:46:42.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.132306501Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-16T08:46:42.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.132568517Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-16T08:46:42.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.132956429Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T08:46:42.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.133209059Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T08:46:42.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.133661009Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-16T08:46:42.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.134212865Z level=info msg="Migration successfully executed" id="Drop old dashboard public config table" duration=551.597µs 2024-09-16T08:46:42.304 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.134577967Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-16T08:46:42.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.135246446Z level=info msg="Migration successfully executed" id="recreate dashboard public config v1" duration=668.325µs 2024-09-16T08:46:42.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.135608476Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-16T08:46:42.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.136272703Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_uid - v1" duration=664.063µs 2024-09-16T08:46:42.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.136633667Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T08:46:42.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.137333543Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" duration=698.538µs 2024-09-16T08:46:42.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.137684895Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-16T08:46:42.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.138339845Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_public_config_uid - v2" duration=654.726µs 2024-09-16T08:46:42.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.138674932Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-16T08:46:42.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.139374584Z level=info msg="Migration successfully executed" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=699.557µs 2024-09-16T08:46:42.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.139729786Z level=info msg="Executing migration" id="Drop public config table" 2024-09-16T08:46:42.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.140274041Z level=info msg="Migration successfully executed" id="Drop public config table" duration=544.042µs 2024-09-16T08:46:42.305 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.140637483Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-16T08:46:42.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.141328504Z level=info msg="Migration successfully executed" id="Recreate dashboard public config v2" duration=690.891µs 2024-09-16T08:46:42.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.141683224Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-16T08:46:42.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.142356933Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_uid - v2" duration=673.354µs 2024-09-16T08:46:42.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.142709732Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-16T08:46:42.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.143408229Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=698.393µs 2024-09-16T08:46:42.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.143760492Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-16T08:46:42.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.144498846Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_access_token - v2" duration=738.269µs 2024-09-16T08:46:42.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.144868235Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-16T08:46:42.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.161488643Z level=info msg="Migration successfully executed" id="Rename table dashboard_public_config to dashboard_public - v2" duration=16.616687ms 2024-09-16T08:46:42.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.161892871Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-16T08:46:42.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.166954999Z level=info msg="Migration successfully executed" id="add annotations_enabled column" duration=5.061908ms 2024-09-16T08:46:42.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.167307183Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-16T08:46:42.306 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.1720984Z level=info msg="Migration successfully executed" id="add time_selection_enabled column" duration=4.790965ms 2024-09-16T08:46:42.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.172456254Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-16T08:46:42.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.172597512Z level=info msg="Migration successfully executed" id="delete orphaned public dashboards" duration=141.312µs 2024-09-16T08:46:42.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.175937495Z level=info msg="Executing migration" id="add share column" 2024-09-16T08:46:42.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.180703764Z level=info msg="Migration successfully executed" id="add share column" duration=4.764557ms 2024-09-16T08:46:42.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.181069212Z level=info msg="Executing migration" id="backfill empty share column fields with default of public" 2024-09-16T08:46:42.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.181201Z level=info msg="Migration successfully executed" id="backfill empty share column fields with default of public" duration=130.153µs 2024-09-16T08:46:42.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.184064375Z level=info msg="Executing migration" id="create file table" 2024-09-16T08:46:42.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.184668338Z level=info msg="Migration successfully executed" id="create file table" duration=603.868µs 2024-09-16T08:46:42.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.186805842Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-16T08:46:42.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.187469398Z level=info msg="Migration successfully executed" id="file table idx: path natural pk" duration=663.538µs 2024-09-16T08:46:42.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.189434673Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-16T08:46:42.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.19020599Z level=info msg="Migration successfully executed" id="file table idx: parent_folder_path_hash fast folder retrieval" duration=773.569µs 2024-09-16T08:46:42.307 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.191961556Z level=info msg="Executing migration" id="create file_meta table" 2024-09-16T08:46:42.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.192459027Z level=info msg="Migration successfully executed" id="create file_meta table" duration=502.771µs 2024-09-16T08:46:42.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.19463841Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-16T08:46:42.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.195323106Z level=info msg="Migration successfully executed" id="file table idx: path key" duration=684.302µs 2024-09-16T08:46:42.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.197440119Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-16T08:46:42.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.197524492Z level=info msg="Migration successfully executed" id="set path collation in file table" duration=86.844µs 2024-09-16T08:46:42.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.200031966Z level=info msg="Executing migration" id="migrate contents column to mediumblob for MySQL" 2024-09-16T08:46:42.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.200102812Z level=info msg="Migration successfully executed" id="migrate contents column to mediumblob for MySQL" duration=71.228µs 2024-09-16T08:46:42.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.202470974Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-16T08:46:42.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.202799925Z level=info msg="Migration successfully executed" id="managed permissions migration" duration=328.749µs 2024-09-16T08:46:42.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.205130679Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-16T08:46:42.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.205417141Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions migration" duration=288.724µs 2024-09-16T08:46:42.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.207647281Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-16T08:46:42.308 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.208455011Z level=info msg="Migration successfully executed" id="RBAC action name migrator" duration=807.457µs 2024-09-16T08:46:42.309 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.210353336Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-16T08:46:42.309 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.215279043Z level=info msg="Migration successfully executed" id="Add UID column to playlist" duration=4.924238ms 2024-09-16T08:46:42.309 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.215639576Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-16T08:46:42.309 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.21574144Z level=info msg="Migration successfully executed" id="Update uid column values in playlist" duration=101.967µs 2024-09-16T08:46:42.309 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.218390571Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-16T08:46:42.309 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.219138114Z level=info msg="Migration successfully executed" id="Add index for uid in playlist" duration=747.247µs 2024-09-16T08:46:42.309 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.221290323Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-16T08:46:42.309 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.221640122Z level=info msg="Migration successfully executed" id="update group index for alert rules" duration=352.953µs 2024-09-16T08:46:42.309 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.223878987Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-16T08:46:42.309 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.224107771Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions repeated migration" duration=230.279µs 2024-09-16T08:46:42.309 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.228950226Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-16T08:46:42.309 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.22925374Z level=info msg="Migration successfully executed" id="admin only folder/dashboard permission" duration=306.287µs 2024-09-16T08:46:42.309 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.229822772Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-16T08:46:42.310 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.234827684Z level=info msg="Migration successfully executed" id="add action column to seed_assignment" duration=5.006841ms 2024-09-16T08:46:42.310 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.235179907Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-16T08:46:42.310 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.240059195Z level=info msg="Migration successfully executed" id="add scope column to seed_assignment" duration=4.879208ms 2024-09-16T08:46:42.310 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.240419947Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-16T08:46:42.310 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.24109085Z level=info msg="Migration successfully executed" id="remove unique index builtin_role_role_name before nullable update" duration=670.836µs 2024-09-16T08:46:42.310 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.241440701Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-16T08:46:42.310 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.29650746Z level=info msg="Migration successfully executed" id="update seed_assignment role_name column to nullable" duration=55.066355ms 2024-09-16T08:46:42.310 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.296930164Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-16T08:46:42.310 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.297767033Z level=info msg="Migration successfully executed" id="add unique index builtin_role_name back" duration=831.052µs 2024-09-16T08:46:42.310 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.298155341Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-16T08:46:42.310 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.299025737Z level=info msg="Migration successfully executed" id="add unique index builtin_role_action_scope" duration=870.783µs 2024-09-16T08:46:42.310 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.299438083Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-16T08:46:42.554 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.317707221Z level=info msg="Migration successfully executed" id="add primary key to seed_assigment" duration=18.27131ms 2024-09-16T08:46:42.554 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.318100018Z level=info msg="Executing migration" id="add origin column to seed_assignment" 2024-09-16T08:46:42.554 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.323162799Z level=info msg="Migration successfully executed" id="add origin column to seed_assignment" duration=5.062523ms 2024-09-16T08:46:42.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.323604816Z level=info msg="Executing migration" id="add origin to plugin seed_assignment" 2024-09-16T08:46:42.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.323798075Z level=info msg="Migration successfully executed" id="add origin to plugin seed_assignment" duration=193.375µs 2024-09-16T08:46:42.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.324443056Z level=info msg="Executing migration" id="prevent seeding OnCall access" 2024-09-16T08:46:42.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.324539532Z level=info msg="Migration successfully executed" id="prevent seeding OnCall access" duration=96.731µs 2024-09-16T08:46:42.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.325453095Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-16T08:46:42.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.325700051Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions repeated fixed migration" duration=248.012µs 2024-09-16T08:46:42.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.326135474Z level=info msg="Executing migration" id="managed folder permissions library panel actions migration" 2024-09-16T08:46:42.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.3262571Z level=info msg="Migration successfully executed" id="managed folder permissions library panel actions migration" duration=121.652µs 2024-09-16T08:46:42.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.326629388Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-16T08:46:42.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.326748625Z level=info msg="Migration successfully executed" id="migrate external alertmanagers to datsourcse" duration=119.26µs 2024-09-16T08:46:42.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.327515703Z level=info msg="Executing migration" id="create folder table" 2024-09-16T08:46:42.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.328102618Z level=info msg="Migration successfully executed" id="create folder table" duration=586.891µs 2024-09-16T08:46:42.555 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.328454454Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-16T08:46:42.556 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.329246034Z level=info msg="Migration successfully executed" id="Add index for parent_uid" duration=791.246µs 2024-09-16T08:46:42.556 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.32961406Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-16T08:46:42.556 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.330332327Z level=info msg="Migration successfully executed" id="Add unique index for folder.uid and folder.org_id" duration=717.997µs 2024-09-16T08:46:42.556 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.330710476Z level=info msg="Executing migration" id="Update folder title length" 2024-09-16T08:46:42.556 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.33072646Z level=info msg="Migration successfully executed" id="Update folder title length" duration=16.601µs 2024-09-16T08:46:42.557 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.331547922Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-16T08:46:42.557 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.332311364Z level=info msg="Migration successfully executed" id="Add unique index for folder.title and folder.parent_uid" duration=762.929µs 2024-09-16T08:46:42.557 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.332664821Z level=info msg="Executing migration" id="Remove unique index for folder.title and folder.parent_uid" 2024-09-16T08:46:42.557 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.333344228Z level=info msg="Migration successfully executed" id="Remove unique index for folder.title and folder.parent_uid" duration=680.457µs 2024-09-16T08:46:42.557 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.333709944Z level=info msg="Executing migration" id="Add unique index for title, parent_uid, and org_id" 2024-09-16T08:46:42.557 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.334436269Z level=info msg="Migration successfully executed" id="Add unique index for title, parent_uid, and org_id" duration=739.81µs 2024-09-16T08:46:42.557 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.334802036Z level=info msg="Executing migration" id="Sync dashboard and folder table" 2024-09-16T08:46:42.557 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.335089741Z level=info msg="Migration successfully executed" id="Sync dashboard and folder table" duration=287.667µs 2024-09-16T08:46:42.557 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.335667344Z level=info msg="Executing migration" id="Remove ghost folders from the folder table" 2024-09-16T08:46:42.557 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.335850969Z level=info msg="Migration successfully executed" id="Remove ghost folders from the folder table" duration=183.519µs 2024-09-16T08:46:42.557 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.33654167Z level=info msg="Executing migration" id="Remove unique index UQE_folder_uid_org_id" 2024-09-16T08:46:42.557 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.337223413Z level=info msg="Migration successfully executed" id="Remove unique index UQE_folder_uid_org_id" duration=681.657µs 2024-09-16T08:46:42.558 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.3375737Z level=info msg="Executing migration" id="Add unique index UQE_folder_org_id_uid" 2024-09-16T08:46:42.558 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.338275101Z level=info msg="Migration successfully executed" id="Add unique index UQE_folder_org_id_uid" duration=701.563µs 2024-09-16T08:46:42.558 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.338611638Z level=info msg="Executing migration" id="Remove unique index UQE_folder_title_parent_uid_org_id" 2024-09-16T08:46:42.558 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.33926803Z level=info msg="Migration successfully executed" id="Remove unique index UQE_folder_title_parent_uid_org_id" duration=657.551µs 2024-09-16T08:46:42.558 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.339604093Z level=info msg="Executing migration" id="Add unique index UQE_folder_org_id_parent_uid_title" 2024-09-16T08:46:42.558 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.340355984Z level=info msg="Migration successfully executed" id="Add unique index UQE_folder_org_id_parent_uid_title" duration=751.576µs 2024-09-16T08:46:42.558 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.340704947Z level=info msg="Executing migration" id="Remove index IDX_folder_parent_uid_org_id" 2024-09-16T08:46:42.558 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.341378902Z level=info msg="Migration successfully executed" id="Remove index IDX_folder_parent_uid_org_id" duration=673.597µs 2024-09-16T08:46:42.558 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.341733032Z level=info msg="Executing migration" id="create anon_device table" 2024-09-16T08:46:42.558 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.342296927Z level=info msg="Migration successfully executed" id="create anon_device table" duration=563.678µs 2024-09-16T08:46:42.558 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.342673263Z level=info msg="Executing migration" id="add unique index anon_device.device_id" 2024-09-16T08:46:42.560 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.343437038Z level=info msg="Migration successfully executed" id="add unique index anon_device.device_id" duration=761.974µs 2024-09-16T08:46:42.560 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.343814619Z level=info msg="Executing migration" id="add index anon_device.updated_at" 2024-09-16T08:46:42.560 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.344515188Z level=info msg="Migration successfully executed" id="add index anon_device.updated_at" duration=700.365µs 2024-09-16T08:46:42.560 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.344878596Z level=info msg="Executing migration" id="create signing_key table" 2024-09-16T08:46:42.560 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.345493234Z level=info msg="Migration successfully executed" id="create signing_key table" duration=615.967µs 2024-09-16T08:46:42.560 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.345848775Z level=info msg="Executing migration" id="add unique index signing_key.key_id" 2024-09-16T08:46:42.560 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.346526775Z level=info msg="Migration successfully executed" id="add unique index signing_key.key_id" duration=678.049µs 2024-09-16T08:46:42.561 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.346881799Z level=info msg="Executing migration" id="set legacy alert migration status in kvstore" 2024-09-16T08:46:42.561 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.347554661Z level=info msg="Migration successfully executed" id="set legacy alert migration status in kvstore" duration=672.899µs 2024-09-16T08:46:42.561 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.347893143Z level=info msg="Executing migration" id="migrate record of created folders during legacy migration to kvstore" 2024-09-16T08:46:42.561 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.348055884Z level=info msg="Migration successfully executed" id="migrate record of created folders during legacy migration to kvstore" duration=163.334µs 2024-09-16T08:46:42.561 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.348748578Z level=info msg="Executing migration" id="Add folder_uid for dashboard" 2024-09-16T08:46:42.561 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.354484601Z level=info msg="Migration successfully executed" id="Add folder_uid for dashboard" duration=5.735781ms 2024-09-16T08:46:42.561 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.354861798Z level=info msg="Executing migration" id="Populate dashboard folder_uid column" 2024-09-16T08:46:42.561 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.355319924Z level=info msg="Migration successfully executed" id="Populate dashboard folder_uid column" duration=458.731µs 2024-09-16T08:46:42.561 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.355728445Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_folder_uid_title" 2024-09-16T08:46:42.561 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.356439676Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_folder_uid_title" duration=709.471µs 2024-09-16T08:46:42.561 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.358851723Z level=info msg="Executing migration" id="Delete unique index for dashboard_org_id_folder_id_title" 2024-09-16T08:46:42.562 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.35952531Z level=info msg="Migration successfully executed" id="Delete unique index for dashboard_org_id_folder_id_title" duration=673.657µs 2024-09-16T08:46:42.562 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.361602509Z level=info msg="Executing migration" id="Delete unique index for dashboard_org_id_folder_uid_title" 2024-09-16T08:46:42.562 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.362298992Z level=info msg="Migration successfully executed" id="Delete unique index for dashboard_org_id_folder_uid_title" duration=696.318µs 2024-09-16T08:46:42.562 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.364444776Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" 2024-09-16T08:46:42.562 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.365192894Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" duration=747.836µs 2024-09-16T08:46:42.562 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.365544386Z level=info msg="Executing migration" id="Restore index for dashboard_org_id_folder_id_title" 2024-09-16T08:46:42.562 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.366243972Z level=info msg="Migration successfully executed" id="Restore index for dashboard_org_id_folder_id_title" duration=699.251µs 2024-09-16T08:46:42.563 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.368441434Z level=info msg="Executing migration" id="create sso_setting table" 2024-09-16T08:46:42.563 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.369112134Z level=info msg="Migration successfully executed" id="create sso_setting table" duration=670.329µs 2024-09-16T08:46:42.563 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.371170639Z level=info msg="Executing migration" id="copy kvstore migration status to each org" 2024-09-16T08:46:42.563 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.371647036Z level=info msg="Migration successfully executed" id="copy kvstore migration status to each org" duration=477.062µs 2024-09-16T08:46:42.563 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.374109438Z level=info msg="Executing migration" id="add back entry for orgid=0 migrated status" 2024-09-16T08:46:42.563 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.374296276Z level=info msg="Migration successfully executed" id="add back entry for orgid=0 migrated status" duration=187.385µs 2024-09-16T08:46:42.563 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.376963724Z level=info msg="Executing migration" id="alter kv_store.value to longtext" 2024-09-16T08:46:42.563 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.377047152Z level=info msg="Migration successfully executed" id="alter kv_store.value to longtext" duration=86.398µs 2024-09-16T08:46:42.563 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.377475396Z level=info msg="Executing migration" id="add notification_settings column to alert_rule table" 2024-09-16T08:46:42.563 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.382597846Z level=info msg="Migration successfully executed" id="add notification_settings column to alert_rule table" duration=5.121152ms 2024-09-16T08:46:42.564 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.382965233Z level=info msg="Executing migration" id="add notification_settings column to alert_rule_version table" 2024-09-16T08:46:42.564 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.388297408Z level=info msg="Migration successfully executed" id="add notification_settings column to alert_rule_version table" duration=5.332018ms 2024-09-16T08:46:42.564 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.388645895Z level=info msg="Executing migration" id="removing scope from alert.instances:read action migration" 2024-09-16T08:46:42.564 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.388860189Z level=info msg="Migration successfully executed" id="removing scope from alert.instances:read action migration" duration=214.386µs 2024-09-16T08:46:42.564 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=migrator t=2024-09-16T08:46:42.389243657Z level=info msg="migrations completed" performed=547 skipped=0 duration=1.227815033s 2024-09-16T08:46:42.564 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=sqlstore t=2024-09-16T08:46:42.390078195Z level=info msg="Created default organization" 2024-09-16T08:46:42.564 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=secrets t=2024-09-16T08:46:42.390582543Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-16T08:46:42.564 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=plugin.store t=2024-09-16T08:46:42.408736339Z level=info msg="Loading plugins..." 2024-09-16T08:46:42.564 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=local.finder t=2024-09-16T08:46:42.44783564Z level=warn msg="Skipping finding plugins as directory does not exist" path=/usr/share/grafana/plugins-bundled 2024-09-16T08:46:42.564 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=plugin.store t=2024-09-16T08:46:42.447853947Z level=info msg="Plugins loaded" count=55 duration=39.118303ms 2024-09-16T08:46:42.564 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=query_data t=2024-09-16T08:46:42.449996851Z level=info msg="Query Service initialization" 2024-09-16T08:46:42.564 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=live.push_http t=2024-09-16T08:46:42.45251574Z level=info msg="Live Push Gateway initialization" 2024-09-16T08:46:42.564 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=ngalert.migration t=2024-09-16T08:46:42.453522473Z level=info msg=Starting 2024-09-16T08:46:42.565 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=ngalert.migration t=2024-09-16T08:46:42.453832183Z level=info msg="Applying transition" currentType=Legacy desiredType=UnifiedAlerting cleanOnDowngrade=false cleanOnUpgrade=false 2024-09-16T08:46:42.565 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=ngalert.migration orgID=1 t=2024-09-16T08:46:42.454093103Z level=info msg="Migrating alerts for organisation" 2024-09-16T08:46:42.565 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=ngalert.migration orgID=1 t=2024-09-16T08:46:42.45451214Z level=info msg="Alerts found to migrate" alerts=0 2024-09-16T08:46:42.565 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=ngalert.migration t=2024-09-16T08:46:42.455633982Z level=info msg="Completed alerting migration" 2024-09-16T08:46:42.565 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=ngalert.state.manager t=2024-09-16T08:46:42.466256433Z level=info msg="Running in alternative execution of Error/NoData mode" 2024-09-16T08:46:42.565 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=infra.usagestats.collector t=2024-09-16T08:46:42.467537336Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-16T08:46:42.565 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=provisioning.datasources t=2024-09-16T08:46:42.469142777Z level=info msg="inserting datasource from configuration" name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-16T08:46:42.565 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=provisioning.datasources t=2024-09-16T08:46:42.481335605Z level=info msg="inserting datasource from configuration" name=Loki uid=P8E80F9AEF21F6940 2024-09-16T08:46:42.565 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=provisioning.alerting t=2024-09-16T08:46:42.494134443Z level=info msg="starting to provision alerting" 2024-09-16T08:46:42.565 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=provisioning.alerting t=2024-09-16T08:46:42.494145247Z level=info msg="finished to provision alerting" 2024-09-16T08:46:42.565 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=ngalert.state.manager t=2024-09-16T08:46:42.494257062Z level=info msg="Warming state cache for startup" 2024-09-16T08:46:42.565 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=ngalert.multiorg.alertmanager t=2024-09-16T08:46:42.494408329Z level=info msg="Starting MultiOrg Alertmanager" 2024-09-16T08:46:42.565 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=grafanaStorageLogger t=2024-09-16T08:46:42.494908546Z level=info msg="Storage starting" 2024-09-16T08:46:42.566 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=http.server t=2024-09-16T08:46:42.497929033Z level=info msg="HTTP Server TLS settings" MinTLSVersion=TLS1.2 configuredciphers=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA 2024-09-16T08:46:42.566 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=http.server t=2024-09-16T08:46:42.49824125Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-16T08:46:42.566 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=provisioning.dashboard t=2024-09-16T08:46:42.519416625Z level=info msg="starting to provision dashboards" 2024-09-16T08:46:42.566 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=ngalert.state.manager t=2024-09-16T08:46:42.519636193Z level=info msg="State cache has been initialized" states=0 duration=25.378201ms 2024-09-16T08:46:42.566 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=ngalert.scheduler t=2024-09-16T08:46:42.519675362Z level=info msg="Starting scheduler" tickInterval=10s maxAttempts=1 2024-09-16T08:46:42.566 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=ticker t=2024-09-16T08:46:42.519720162Z level=info msg=starting first_tick=2024-09-16T08:46:50Z 2024-09-16T08:46:42.566 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=sqlstore.transactions t=2024-09-16T08:46:42.522959395Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T08:46:42.566 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=sqlstore.transactions t=2024-09-16T08:46:42.53387123Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-16T08:46:42.566 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=sqlstore.transactions t=2024-09-16T08:46:42.554100338Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T08:46:42.697 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:42 smithi160 bash[74150]: Copying blob sha256:2abcce694348cd2c949c0e98a7400ebdfd8341021bcf6b541bc72033ce982510 2024-09-16T08:46:42.849 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=sqlstore.transactions t=2024-09-16T08:46:42.566346539Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-16T08:46:42.849 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=sqlstore.transactions t=2024-09-16T08:46:42.577246916Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=2 code="database is locked" 2024-09-16T08:46:42.849 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=plugins.update.checker t=2024-09-16T08:46:42.58944636Z level=info msg="Update check succeeded" duration=95.151153ms 2024-09-16T08:46:42.850 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=grafana.update.checker t=2024-09-16T08:46:42.593214725Z level=info msg="Update check succeeded" duration=98.94334ms 2024-09-16T08:46:42.850 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=sqlstore.transactions t=2024-09-16T08:46:42.610008287Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T08:46:42.850 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=sqlstore.transactions t=2024-09-16T08:46:42.620409309Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-16T08:46:42.850 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=sqlstore.transactions t=2024-09-16T08:46:42.630776507Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=2 code="database is locked" 2024-09-16T08:46:42.850 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=provisioning.dashboard t=2024-09-16T08:46:42.68160056Z level=info msg="finished to provision dashboards" 2024-09-16T08:46:43.349 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=grafana-apiserver t=2024-09-16T08:46:42.867217204Z level=info msg="Adding GroupVersion playlist.grafana.app v0alpha1 to ResourceManager" 2024-09-16T08:46:43.349 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:46:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=grafana-apiserver t=2024-09-16T08:46:42.867611586Z level=info msg="Adding GroupVersion featuretoggle.grafana.app v0alpha1 to ResourceManager" 2024-09-16T08:46:44.197 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:43 smithi160 bash[74150]: Copying config sha256:72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e 2024-09-16T08:46:44.197 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:43 smithi160 bash[74150]: Writing manifest to image destination 2024-09-16T08:46:44.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:44 smithi169 ceph-mon[33465]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:44 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:44.637 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-mon[30137]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:44.637 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:44.637 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-mon[40028]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:44.637 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:44.637 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 podman[74150]: 2024-09-16 08:46:44.25648546 +0000 UTC m=+2.249218044 image pull 72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e quay.io/prometheus/node-exporter:v1.7.0 2024-09-16T08:46:44.637 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 podman[74150]: 2024-09-16 08:46:44.371453362 +0000 UTC m=+2.364185934 container create 88d23ccb71c1cd7f91ff4e42b02f7e4ab24d2b1a75cbd28aca32d67d88970127 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T08:46:44.948 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 podman[74150]: 2024-09-16 08:46:44.63711274 +0000 UTC m=+2.629845311 container init 88d23ccb71c1cd7f91ff4e42b02f7e4ab24d2b1a75cbd28aca32d67d88970127 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T08:46:44.948 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 podman[74150]: 2024-09-16 08:46:44.642196452 +0000 UTC m=+2.634929023 container start 88d23ccb71c1cd7f91ff4e42b02f7e4ab24d2b1a75cbd28aca32d67d88970127 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T08:46:44.948 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.645Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)" 2024-09-16T08:46:44.948 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.645Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)" 2024-09-16T08:46:44.948 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.646Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-16T08:46:44.948 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.646Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-16T08:46:44.948 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.646Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-16T08:46:44.948 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.646Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-16T08:46:44.948 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-16T08:46:44.949 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=arp 2024-09-16T08:46:44.949 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-16T08:46:44.949 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-16T08:46:44.949 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-16T08:46:44.949 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-16T08:46:44.949 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-16T08:46:44.949 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-16T08:46:44.950 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-16T08:46:44.950 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-16T08:46:44.950 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=edac 2024-09-16T08:46:44.950 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-16T08:46:44.950 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-16T08:46:44.950 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-16T08:46:44.950 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-16T08:46:44.950 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-16T08:46:44.950 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-16T08:46:44.950 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-16T08:46:44.951 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-16T08:46:44.951 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-16T08:46:44.951 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-16T08:46:44.951 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-16T08:46:44.951 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-16T08:46:44.951 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-16T08:46:44.951 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-16T08:46:44.951 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-16T08:46:44.951 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-16T08:46:44.951 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=os 2024-09-16T08:46:44.951 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-16T08:46:44.951 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-16T08:46:44.951 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-16T08:46:44.952 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-16T08:46:44.952 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-16T08:46:44.952 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-16T08:46:44.952 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-16T08:46:44.952 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=stat 2024-09-16T08:46:44.952 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-16T08:46:44.952 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-16T08:46:44.952 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-16T08:46:44.952 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=time 2024-09-16T08:46:44.952 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-16T08:46:44.952 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=uname 2024-09-16T08:46:44.952 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-16T08:46:44.953 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-16T08:46:44.953 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-16T08:46:44.953 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100 2024-09-16T08:46:44.953 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a[74224]: ts=2024-09-16T08:46:44.647Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-16T08:46:44.953 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 bash[74150]: 88d23ccb71c1cd7f91ff4e42b02f7e4ab24d2b1a75cbd28aca32d67d88970127 2024-09-16T08:46:44.953 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 08:46:44 smithi160 systemd[1]: Started Ceph node-exporter.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:46:45.099 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:46:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:46:45.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:46:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:46:45.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:45 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:45.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:45 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:45.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:45 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:45.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:45 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:45.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:45 smithi169 ceph-mon[33465]: Deploying daemon node-exporter.b on smithi169 2024-09-16T08:46:45.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:45 smithi169 ceph-mon[33465]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:46.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:45 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:46.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:45 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:46.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:45 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:46.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:45 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:46.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:45 smithi160 ceph-mon[40028]: Deploying daemon node-exporter.b on smithi169 2024-09-16T08:46:46.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:45 smithi160 ceph-mon[40028]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:46.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:45 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:46.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:45 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:46.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:45 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:46.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:45 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:46.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:45 smithi160 ceph-mon[30137]: Deploying daemon node-exporter.b on smithi169 2024-09-16T08:46:46.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:45 smithi160 ceph-mon[30137]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:46.599 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:46 smithi169 bash[63725]: Copying blob sha256:455fd88e5221bc1e278ef2d059cd70e4df99a24e5af050ede621534276f6cf9a 2024-09-16T08:46:47.792 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:47 smithi169 bash[63725]: Copying config sha256:72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e 2024-09-16T08:46:47.792 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:47 smithi169 bash[63725]: Writing manifest to image destination 2024-09-16T08:46:48.090 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:47 smithi169 podman[63725]: 2024-09-16 08:46:47.73109226 +0000 UTC m=+2.062019094 image pull 72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e quay.io/prometheus/node-exporter:v1.7.0 2024-09-16T08:46:48.090 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:47 smithi169 podman[63725]: 2024-09-16 08:46:47.829616654 +0000 UTC m=+2.160543476 container create 8b16f01ba1fe7784d0c5df1548d6b86c05eab1e542adadc763add53a6fffd305 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T08:46:48.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-mon[33465]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:48.350 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 podman[63725]: 2024-09-16 08:46:48.090504475 +0000 UTC m=+2.421431297 container init 8b16f01ba1fe7784d0c5df1548d6b86c05eab1e542adadc763add53a6fffd305 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T08:46:48.350 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 podman[63725]: 2024-09-16 08:46:48.096113314 +0000 UTC m=+2.427040139 container start 8b16f01ba1fe7784d0c5df1548d6b86c05eab1e542adadc763add53a6fffd305 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T08:46:48.350 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.100Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)" 2024-09-16T08:46:48.350 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.100Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)" 2024-09-16T08:46:48.350 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.101Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-16T08:46:48.350 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.101Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-16T08:46:48.350 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.101Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-16T08:46:48.350 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.101Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-16T08:46:48.350 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-16T08:46:48.351 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=arp 2024-09-16T08:46:48.351 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-16T08:46:48.351 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-16T08:46:48.351 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-16T08:46:48.351 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-16T08:46:48.351 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-16T08:46:48.351 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-16T08:46:48.351 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-16T08:46:48.351 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-16T08:46:48.351 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=edac 2024-09-16T08:46:48.351 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-16T08:46:48.351 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-16T08:46:48.351 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-16T08:46:48.352 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-16T08:46:48.352 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-16T08:46:48.352 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-16T08:46:48.352 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-16T08:46:48.352 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.102Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-16T08:46:48.352 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-16T08:46:48.352 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-16T08:46:48.352 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-16T08:46:48.352 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-16T08:46:48.352 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-16T08:46:48.352 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-16T08:46:48.352 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-16T08:46:48.352 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-16T08:46:48.352 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=os 2024-09-16T08:46:48.353 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-16T08:46:48.353 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-16T08:46:48.353 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-16T08:46:48.353 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-16T08:46:48.353 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-16T08:46:48.353 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-16T08:46:48.353 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-16T08:46:48.353 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=stat 2024-09-16T08:46:48.353 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-16T08:46:48.353 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-16T08:46:48.354 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-16T08:46:48.354 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=time 2024-09-16T08:46:48.354 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-16T08:46:48.354 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=uname 2024-09-16T08:46:48.354 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-16T08:46:48.354 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-16T08:46:48.354 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-16T08:46:48.354 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100 2024-09-16T08:46:48.354 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b[63799]: ts=2024-09-16T08:46:48.103Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-16T08:46:48.354 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 bash[63725]: 8b16f01ba1fe7784d0c5df1548d6b86c05eab1e542adadc763add53a6fffd305 2024-09-16T08:46:48.354 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 08:46:48 smithi169 systemd[1]: Started Ceph node-exporter.b for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:46:48.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:48 smithi160 ceph-mon[30137]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:48.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:48 smithi160 ceph-mon[40028]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:49.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:46:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:46:49.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:46:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:46:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:46:49.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:49.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:46:50.406 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:50 smithi160 ceph-mon[30137]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:50.407 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:50 smithi160 ceph-mon[40028]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:50.675 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:50 smithi169 ceph-mon[33465]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:52.804 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:52 smithi160 ceph-mon[30137]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:52.804 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:52 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:52.805 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:52 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:52.805 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:52 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:52.805 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:52 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:52.805 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:52 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:46:52.805 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:52 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:46:52.805 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:52 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:52.805 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:52 smithi160 ceph-mon[40028]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:52.805 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:52 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:52.805 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:52 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:52.805 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:52 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:52.806 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:52 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:52.806 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:52 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:46:52.806 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:52 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:46:52.806 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:52 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:52.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:52 smithi169 ceph-mon[33465]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:52.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:52 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:52.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:52 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:52.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:52 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:52 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:52 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:46:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:52 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:46:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:52 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:46:53.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:53 smithi169 ceph-mon[33465]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-16T08:46:53.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:53 smithi169 ceph-mon[33465]: Reconfiguring daemon alertmanager.a on smithi160 2024-09-16T08:46:53.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:53 smithi160 ceph-mon[30137]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-16T08:46:53.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:53 smithi160 ceph-mon[30137]: Reconfiguring daemon alertmanager.a on smithi160 2024-09-16T08:46:53.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:53 smithi160 ceph-mon[40028]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-16T08:46:53.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:53 smithi160 ceph-mon[40028]: Reconfiguring daemon alertmanager.a on smithi160 2024-09-16T08:46:54.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:54 smithi169 ceph-mon[33465]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:54.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:46:54 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:46:54.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:54 smithi160 ceph-mon[30137]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:54.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:54 smithi160 ceph-mon[40028]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:55.345 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:46:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:46:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:46:56.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:56 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:56.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:56 smithi169 ceph-mon[33465]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:56.887 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:56 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:56.887 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:56 smithi160 ceph-mon[30137]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:56.887 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:56 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:46:56.887 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:56 smithi160 ceph-mon[40028]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:46:58.139 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:57 smithi160 systemd[1]: Stopping Ceph alertmanager.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T08:46:58.422 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:58 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[73896]: ts=2024-09-16T08:46:58.138Z caller=main.go:542 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-16T08:46:58.422 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:58 smithi160 podman[74930]: 2024-09-16 08:46:58.169625911 +0000 UTC m=+0.152464994 container died acebf1ef0331f6201baeb71373f5cc3de1aa748b5649bf4b6a35bfe9d5884411 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:46:58.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:46:58 smithi160 ceph-mon[30137]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:58.698 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:58 smithi160 podman[74930]: 2024-09-16 08:46:58.520434867 +0000 UTC m=+0.503273956 container cleanup acebf1ef0331f6201baeb71373f5cc3de1aa748b5649bf4b6a35bfe9d5884411 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:46:58.698 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:58 smithi160 bash[74930]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a 2024-09-16T08:46:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:46:58 smithi160 ceph-mon[40028]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:58.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:46:58 smithi169 ceph-mon[33465]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:46:59.198 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:58 smithi160 podman[74946]: 2024-09-16 08:46:58.830445705 +0000 UTC m=+0.663436967 container remove acebf1ef0331f6201baeb71373f5cc3de1aa748b5649bf4b6a35bfe9d5884411 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:46:59.198 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:58 smithi160 podman[74946]: 2024-09-16 08:46:58.931203947 +0000 UTC m=+0.764195211 volume remove e21079aa62dd775e0215bace7beb4bb8900e7530966bccbce6df24f342e8fddf 2024-09-16T08:46:59.541 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-16T08:46:59.541 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 systemd[1]: Stopped Ceph alertmanager.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:46:59.541 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@alertmanager.a.service: Consumed 1.058s CPU time. 2024-09-16T08:46:59.542 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 systemd[1]: Starting Ceph alertmanager.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T08:46:59.542 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 podman[75037]: 2024-09-16 08:46:59.42420392 +0000 UTC m=+0.100846572 volume create 6c32d6cf87daf28a6ca3640e0f9c8987224e083ee6b970bc1215fb34a23d73d9 2024-09-16T08:46:59.542 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 podman[75037]: 2024-09-16 08:46:59.341190151 +0000 UTC m=+0.017832808 image pull 11f11916f8cdfb17c1da74b20deb2075d778357b4d60474d82453303f8d461f5 quay.io/prometheus/alertmanager:v0.27.0 2024-09-16T08:46:59.542 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 podman[75037]: 2024-09-16 08:46:59.541318235 +0000 UTC m=+0.217960881 container create 6156f45f8ef04943f69f4620eba3a5e176ba24abf2820c0fdc369d7a5e09d162 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:47:00.198 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 podman[75037]: 2024-09-16 08:46:59.845136129 +0000 UTC m=+0.521778784 container init 6156f45f8ef04943f69f4620eba3a5e176ba24abf2820c0fdc369d7a5e09d162 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:47:00.198 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 podman[75037]: 2024-09-16 08:46:59.851952101 +0000 UTC m=+0.528594752 container start 6156f45f8ef04943f69f4620eba3a5e176ba24abf2820c0fdc369d7a5e09d162 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:47:00.198 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T08:46:59.864Z caller=main.go:181 level=info msg="Starting Alertmanager" version="(version=0.27.0, branch=HEAD, revision=0aa3c2aad14cff039931923ab16b26b7481783b5)" 2024-09-16T08:47:00.198 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T08:46:59.864Z caller=main.go:182 level=info build_context="(go=go1.21.7, platform=linux/amd64, user=root@22cd11f671e9, date=20240228-11:51:20, tags=netgo)" 2024-09-16T08:47:00.198 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T08:46:59.865Z caller=cluster.go:186 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.160 port=9094 2024-09-16T08:47:00.198 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T08:46:59.866Z caller=cluster.go:683 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-16T08:47:00.198 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T08:46:59.892Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T08:47:00.199 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T08:46:59.893Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T08:47:00.199 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T08:46:59.896Z caller=tls_config.go:313 level=info msg="Listening on" address=[::]:9093 2024-09-16T08:47:00.199 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T08:46:59.896Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-16T08:47:00.199 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 bash[75037]: 6156f45f8ef04943f69f4620eba3a5e176ba24abf2820c0fdc369d7a5e09d162 2024-09-16T08:47:00.199 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:46:59 smithi160 systemd[1]: Started Ceph alertmanager.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:47:00.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:00 smithi169 ceph-mon[33465]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:47:00.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:00 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:00.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:00 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:00.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:00 smithi160 ceph-mon[30137]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:47:00.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:00 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:00.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:00 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:00.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:00 smithi160 ceph-mon[40028]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:47:00.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:00 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:00.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:00 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:01.534 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:01 smithi160 ceph-mon[30137]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T08:47:01.535 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:01 smithi160 ceph-mon[30137]: Reconfiguring daemon prometheus.a on smithi169 2024-09-16T08:47:01.535 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:01 smithi160 ceph-mon[40028]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T08:47:01.535 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:01 smithi160 ceph-mon[40028]: Reconfiguring daemon prometheus.a on smithi169 2024-09-16T08:47:01.826 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:01 smithi169 ceph-mon[33465]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T08:47:01.827 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:01 smithi169 ceph-mon[33465]: Reconfiguring daemon prometheus.a on smithi169 2024-09-16T08:47:02.200 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:47:01 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T08:47:01.867Z caller=cluster.go:708 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000628723s 2024-09-16T08:47:02.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:02 smithi160 ceph-mon[30137]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:47:02.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:02 smithi160 ceph-mon[40028]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:47:02.809 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:02 smithi169 ceph-mon[33465]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:47:03.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]} : dispatch 2024-09-16T08:47:03.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]} : dispatch 2024-09-16T08:47:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-16T08:47:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:03 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:47:03.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]} : dispatch 2024-09-16T08:47:03.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]} : dispatch 2024-09-16T08:47:03.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-16T08:47:03.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:03 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:47:03.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]} : dispatch 2024-09-16T08:47:03.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]} : dispatch 2024-09-16T08:47:03.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-16T08:47:03.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:03 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:47:04.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:47:04 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:47:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:04 smithi169 ceph-mon[33465]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:47:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-16T08:47:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]': finished 2024-09-16T08:47:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-16T08:47:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:04 smithi169 ceph-mon[33465]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T08:47:04.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:04 smithi160 ceph-mon[30137]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:47:04.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-16T08:47:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]': finished 2024-09-16T08:47:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-16T08:47:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:04 smithi160 ceph-mon[30137]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T08:47:04.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:04 smithi160 ceph-mon[40028]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:47:04.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-16T08:47:04.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]': finished 2024-09-16T08:47:04.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-16T08:47:04.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:04 smithi160 ceph-mon[40028]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T08:47:05.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:47:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:47:05.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:05 smithi169 ceph-mon[33465]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T08:47:05.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:05 smithi160 ceph-mon[30137]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T08:47:05.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:05 smithi160 ceph-mon[40028]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T08:47:06.849 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:06 smithi169 systemd[1]: Stopping Ceph prometheus.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T08:47:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:06 smithi169 ceph-mon[33465]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:47:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:06 smithi169 ceph-mon[33465]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-16T08:47:06.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:06.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:06 smithi160 ceph-mon[30137]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:47:06.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:06 smithi160 ceph-mon[30137]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-16T08:47:06.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:06.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:06 smithi160 ceph-mon[40028]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:47:06.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:06 smithi160 ceph-mon[40028]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-16T08:47:07.248 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:06 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:47:06.957Z caller=main.go:964 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-16T08:47:07.248 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:06 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:47:06.957Z caller=main.go:988 level=info msg="Stopping scrape discovery manager..." 2024-09-16T08:47:07.248 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:06 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:47:06.957Z caller=main.go:1002 level=info msg="Stopping notify discovery manager..." 2024-09-16T08:47:07.248 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:06 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:47:06.957Z caller=manager.go:177 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-16T08:47:07.248 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:06 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:47:06.957Z caller=main.go:984 level=info msg="Scrape discovery manager stopped" 2024-09-16T08:47:07.248 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:06 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:47:06.957Z caller=main.go:998 level=info msg="Notify discovery manager stopped" 2024-09-16T08:47:07.248 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:06 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:47:06.957Z caller=manager.go:187 level=info component="rule manager" msg="Rule manager stopped" 2024-09-16T08:47:07.248 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:06 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:47:06.957Z caller=main.go:1039 level=info msg="Stopping scrape manager..." 2024-09-16T08:47:07.248 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:06 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:47:06.957Z caller=main.go:1031 level=info msg="Scrape manager stopped" 2024-09-16T08:47:07.249 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:06 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:47:06.958Z caller=notifier.go:618 level=info component=notifier msg="Stopping notification manager..." 2024-09-16T08:47:07.249 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:06 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:47:06.958Z caller=main.go:1261 level=info msg="Notifier manager stopped" 2024-09-16T08:47:07.249 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:06 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[61891]: ts=2024-09-16T08:47:06.958Z caller=main.go:1273 level=info msg="See you next time!" 2024-09-16T08:47:07.249 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:06 smithi169 podman[64632]: 2024-09-16 08:47:06.968015551 +0000 UTC m=+0.144129028 container died af370486808d78813b8ae3b6c4654f8e72eb181dd581d16b299312cde6c7866a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:47:07.599 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:07 smithi169 podman[64632]: 2024-09-16 08:47:07.354988759 +0000 UTC m=+0.531102236 container cleanup af370486808d78813b8ae3b6c4654f8e72eb181dd581d16b299312cde6c7866a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:47:07.599 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:07 smithi169 bash[64632]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a 2024-09-16T08:47:07.990 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:07 smithi169 podman[64649]: 2024-09-16 08:47:07.606371675 +0000 UTC m=+0.639858122 container remove af370486808d78813b8ae3b6c4654f8e72eb181dd581d16b299312cde6c7866a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:47:07.990 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:07 smithi169 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-16T08:47:07.990 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:07 smithi169 systemd[1]: Stopped Ceph prometheus.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:47:07.990 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:07 smithi169 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@prometheus.a.service: Consumed 1.073s CPU time. 2024-09-16T08:47:07.990 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:07 smithi169 systemd[1]: Starting Ceph prometheus.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T08:47:08.250 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:07 smithi169 podman[64746]: 2024-09-16 08:47:07.990527854 +0000 UTC m=+0.111179182 container create 45a00bae074a7d5e0d5dee764328e8c530b90aed2f2c41173a382faac3581ea8 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:47:08.250 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:07 smithi169 podman[64746]: 2024-09-16 08:47:07.896469107 +0000 UTC m=+0.017120445 image pull 1d3b7f56885b6dd623f1785be963aa9c195f86bc256ea454e8d02a7980b79c53 quay.io/prometheus/prometheus:v2.51.0 2024-09-16T08:47:08.250 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 podman[64746]: 2024-09-16 08:47:08.201022884 +0000 UTC m=+0.321674216 container init 45a00bae074a7d5e0d5dee764328e8c530b90aed2f2c41173a382faac3581ea8 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:47:08.250 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 podman[64746]: 2024-09-16 08:47:08.205784604 +0000 UTC m=+0.326435934 container start 45a00bae074a7d5e0d5dee764328e8c530b90aed2f2c41173a382faac3581ea8 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:47:08.250 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.225Z caller=main.go:617 level=info msg="Starting Prometheus Server" mode=server version="(version=2.51.0, branch=HEAD, revision=c05c15512acb675e3f6cd662a6727854e93fc024)" 2024-09-16T08:47:08.250 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.225Z caller=main.go:622 level=info build_context="(go=go1.22.1, platform=linux/amd64, user=root@b5723e458358, date=20240319-10:54:45, tags=netgo,builtinassets,stringlabels)" 2024-09-16T08:47:08.250 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.225Z caller=main.go:623 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi169 (none))" 2024-09-16T08:47:08.250 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.225Z caller=main.go:624 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-16T08:47:08.250 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.225Z caller=main.go:625 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-16T08:47:08.250 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.227Z caller=web.go:568 level=info component=web msg="Start listening for connections" address=:9095 2024-09-16T08:47:08.251 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.228Z caller=main.go:1129 level=info msg="Starting TSDB ..." 2024-09-16T08:47:08.251 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.229Z caller=tls_config.go:313 level=info component=web msg="Listening on" address=[::]:9095 2024-09-16T08:47:08.251 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.229Z caller=tls_config.go:316 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-16T08:47:08.251 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.231Z caller=head.go:616 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-16T08:47:08.251 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.231Z caller=head.go:698 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.805µs 2024-09-16T08:47:08.251 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.231Z caller=head.go:706 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-16T08:47:08.251 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.231Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-16T08:47:08.251 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.231Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-16T08:47:08.251 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.231Z caller=head.go:815 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=23.553µs wal_replay_duration=552.971µs wbl_replay_duration=130ns total_replay_duration=595.127µs 2024-09-16T08:47:08.251 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.233Z caller=main.go:1150 level=info fs_type=XFS_SUPER_MAGIC 2024-09-16T08:47:08.251 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.233Z caller=main.go:1153 level=info msg="TSDB started" 2024-09-16T08:47:08.251 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.233Z caller=main.go:1335 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-16T08:47:08.252 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.249Z caller=main.go:1372 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=16.113481ms db_storage=788ns remote_storage=1.151µs web_handler=279ns query_engine=625ns scrape=561.579µs scrape_sd=113.981µs notify=9.575µs notify_sd=8.182µs rules=15.0743ms tracing=5.722µs 2024-09-16T08:47:08.252 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.249Z caller=main.go:1114 level=info msg="Server is ready to receive web requests." 2024-09-16T08:47:08.252 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T08:47:08.249Z caller=manager.go:163 level=info component="rule manager" msg="Starting rule manager..." 2024-09-16T08:47:08.569 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 bash[64746]: 45a00bae074a7d5e0d5dee764328e8c530b90aed2f2c41173a382faac3581ea8 2024-09-16T08:47:08.569 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 08:47:08 smithi169 systemd[1]: Started Ceph prometheus.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T08:47:08.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[30137]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-16T08:47:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-16T08:47:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi160.front.sepia.ceph.com:9093"} : dispatch 2024-09-16T08:47:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-16T08:47:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi169.front.sepia.ceph.com:9095"} : dispatch 2024-09-16T08:47:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-16T08:47:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi169.front.sepia.ceph.com:3000"} : dispatch 2024-09-16T08:47:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:47:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[40028]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-16T08:47:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-16T08:47:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi160.front.sepia.ceph.com:9093"} : dispatch 2024-09-16T08:47:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-16T08:47:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi169.front.sepia.ceph.com:9095"} : dispatch 2024-09-16T08:47:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-16T08:47:08.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi169.front.sepia.ceph.com:3000"} : dispatch 2024-09-16T08:47:08.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:08.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:47:08.700 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:08] ENGINE Bus STOPPING 2024-09-16T08:47:08.700 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:08] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T08:47:08.700 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:08] ENGINE Bus STOPPED 2024-09-16T08:47:08.700 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:08] ENGINE Bus STARTING 2024-09-16T08:47:08.700 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:08] ENGINE Serving on http://:::9283 2024-09-16T08:47:08.700 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:08] ENGINE Bus STARTED 2024-09-16T08:47:08.700 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:08 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:08] ENGINE Bus STOPPING 2024-09-16T08:47:08.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-mon[33465]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-16T08:47:08.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:08.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:08.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-16T08:47:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi160.front.sepia.ceph.com:9093"} : dispatch 2024-09-16T08:47:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-16T08:47:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi169.front.sepia.ceph.com:9095"} : dispatch 2024-09-16T08:47:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-16T08:47:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi169.front.sepia.ceph.com:3000"} : dispatch 2024-09-16T08:47:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:08 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:47:09.347 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:09] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T08:47:09.348 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:09] ENGINE Bus STOPPED 2024-09-16T08:47:09.348 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:09] ENGINE Bus STARTING 2024-09-16T08:47:09.348 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:09] ENGINE Serving on http://:::9283 2024-09-16T08:47:09.348 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:09] ENGINE Bus STARTED 2024-09-16T08:47:09.348 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:09] ENGINE Bus STOPPING 2024-09-16T08:47:09.682 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T08:47:09.682 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi160.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T08:47:09.683 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:47:09.683 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi169.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T08:47:09.683 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T08:47:09.683 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi169.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T08:47:09.683 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:47:09.683 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:47:09.683 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:47:09.683 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:09] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T08:47:09.683 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:09] ENGINE Bus STOPPED 2024-09-16T08:47:09.684 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T08:47:09.684 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi160.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T08:47:09.684 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:47:09.684 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi169.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T08:47:09.684 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T08:47:09.684 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi169.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T08:47:09.684 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:47:09.684 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:47:09.684 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:47:09.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:09 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T08:47:09.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:09 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi160.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T08:47:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:09 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:47:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:09 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi169.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T08:47:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:09 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T08:47:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:09 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi169.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T08:47:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:09 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:47:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:09 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:47:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:09 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:47:09.947 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T08:47:09.869Z caller=cluster.go:700 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002767244s 2024-09-16T08:47:09.948 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:09] ENGINE Bus STARTING 2024-09-16T08:47:09.948 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:09] ENGINE Serving on http://:::9283 2024-09-16T08:47:09.948 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:09 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:08:47:09] ENGINE Bus STARTED 2024-09-16T08:47:10.776 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:10 smithi160 ceph-mon[30137]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-16T08:47:10.776 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:10 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:47:10.776 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:10 smithi160 ceph-mon[30137]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-16T08:47:10.776 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:10 smithi160 ceph-mon[30137]: Cluster is now healthy 2024-09-16T08:47:10.776 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:10 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:47:10.776 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:10 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:47:10.777 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:10 smithi160 ceph-mon[40028]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-16T08:47:10.777 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:10 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:47:10.777 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:10 smithi160 ceph-mon[40028]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-16T08:47:10.777 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:10 smithi160 ceph-mon[40028]: Cluster is now healthy 2024-09-16T08:47:10.777 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:10 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:47:10.777 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:10 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:47:10.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:10 smithi169 ceph-mon[33465]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-16T08:47:10.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:10 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:47:10.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:10 smithi169 ceph-mon[33465]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-16T08:47:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:10 smithi169 ceph-mon[33465]: Cluster is now healthy 2024-09-16T08:47:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:10 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:47:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:10 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:47:11.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:11 smithi160 ceph-mon[30137]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-16T08:47:11.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:11 smithi160 ceph-mon[40028]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-16T08:47:11.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:11 smithi169 ceph-mon[33465]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-16T08:47:12.873 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:12 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:12.873 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:12 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:13.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:12 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:13.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:12 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:13.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:12 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:13.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:12 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:14.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:14 smithi169 ceph-mon[33465]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 23 B/s, 0 objects/s recovering 2024-09-16T08:47:14.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:14 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:14.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:14 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:14.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:14 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:47:14.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:14 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:47:14.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:14 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:14.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:14 smithi160 ceph-mon[30137]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 23 B/s, 0 objects/s recovering 2024-09-16T08:47:14.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:14 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:14.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:14 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:14.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:14 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:47:14.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:14 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:47:14.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:14 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:14 smithi160 ceph-mon[40028]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 23 B/s, 0 objects/s recovering 2024-09-16T08:47:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:14 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:14 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:14 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:47:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:14 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:47:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:14 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:15.081 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:47:14 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:47:15.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:47:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:47:16.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:16.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:16 smithi169 ceph-mon[33465]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 974 B/s rd, 0 op/s; 20 B/s, 0 objects/s recovering 2024-09-16T08:47:16.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:16.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:16 smithi160 ceph-mon[30137]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 974 B/s rd, 0 op/s; 20 B/s, 0 objects/s recovering 2024-09-16T08:47:16.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:16.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:16 smithi160 ceph-mon[40028]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 974 B/s rd, 0 op/s; 20 B/s, 0 objects/s recovering 2024-09-16T08:47:18.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:18 smithi169 ceph-mon[33465]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 0 objects/s recovering 2024-09-16T08:47:18.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:18 smithi160 ceph-mon[30137]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 0 objects/s recovering 2024-09-16T08:47:18.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:18 smithi160 ceph-mon[40028]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 0 objects/s recovering 2024-09-16T08:47:19.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:47:19.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:47:19.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:47:20.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:20 smithi169 ceph-mon[33465]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 18 B/s, 0 objects/s recovering 2024-09-16T08:47:20.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:20 smithi160 ceph-mon[30137]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 18 B/s, 0 objects/s recovering 2024-09-16T08:47:20.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:20 smithi160 ceph-mon[40028]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 18 B/s, 0 objects/s recovering 2024-09-16T08:47:22.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:22 smithi160 ceph-mon[30137]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:47:22.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:22 smithi160 ceph-mon[40028]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:47:22.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:22 smithi169 ceph-mon[33465]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:47:24.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:24 smithi160 ceph-mon[30137]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:47:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:24 smithi160 ceph-mon[40028]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:47:24.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:24 smithi169 ceph-mon[33465]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:47:25.099 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:47:24 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:47:25.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:47:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:47:25.849 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:47:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=infra.usagestats t=2024-09-16T08:47:25.501506746Z level=info msg="Usage stats are ready to report" 2024-09-16T08:47:26.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:26.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:26 smithi160 ceph-mon[40028]: pgmap v46: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:47:26.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:26.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:26 smithi160 ceph-mon[30137]: pgmap v46: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:47:26.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:26.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:26 smithi169 ceph-mon[33465]: pgmap v46: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:47:27.146 INFO:tasks.workunit.client.0.smithi160.stderr:Updating files: 99% (12648/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-16T08:47:27.541 INFO:tasks.workunit.client.0.smithi160.stderr:Note: switching to '8293d73f8690540e843a81caec373f9cc29cf705'. 2024-09-16T08:47:27.541 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-09-16T08:47:27.541 INFO:tasks.workunit.client.0.smithi160.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T08:47:27.541 INFO:tasks.workunit.client.0.smithi160.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T08:47:27.541 INFO:tasks.workunit.client.0.smithi160.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T08:47:27.541 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-09-16T08:47:27.541 INFO:tasks.workunit.client.0.smithi160.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T08:47:27.542 INFO:tasks.workunit.client.0.smithi160.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T08:47:27.542 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-09-16T08:47:27.542 INFO:tasks.workunit.client.0.smithi160.stderr: git switch -c 2024-09-16T08:47:27.542 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-09-16T08:47:27.542 INFO:tasks.workunit.client.0.smithi160.stderr:Or undo this operation with: 2024-09-16T08:47:27.542 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-09-16T08:47:27.542 INFO:tasks.workunit.client.0.smithi160.stderr: git switch - 2024-09-16T08:47:27.542 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-09-16T08:47:27.542 INFO:tasks.workunit.client.0.smithi160.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T08:47:27.542 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-09-16T08:47:27.542 INFO:tasks.workunit.client.0.smithi160.stderr:HEAD is now at 8293d73f869 Merge branch 'wip-aclamk-cbt-combined' of https://github.com/aclamk/ceph into wip-guits-main-2024-09-13-1248 2024-09-16T08:47:27.547 DEBUG:teuthology.orchestra.run.smithi160:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-16T08:47:27.574 INFO:tasks.workunit.client.0.smithi160.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T08:47:27.576 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T08:47:27.576 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T08:47:28.227 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T08:47:28.291 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T08:47:28.330 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T08:47:28.332 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T08:47:28.332 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T08:47:28.371 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T08:47:28.375 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-16T08:47:28.375 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-16T08:47:28.431 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-09-16T08:47:28.432 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-09-16T08:47:28.433 DEBUG:teuthology.orchestra.run.smithi160:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8293d73f8690540e843a81caec373f9cc29cf705 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-09-16T08:47:28.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:28 smithi160 ceph-mon[30137]: pgmap v47: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:47:28.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:28 smithi160 ceph-mon[40028]: pgmap v47: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:47:28.477 INFO:tasks.workunit.client.0.smithi160.stderr:+ parallel=1 2024-09-16T08:47:28.477 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' '' = --serial ']' 2024-09-16T08:47:28.477 INFO:tasks.workunit.client.0.smithi160.stderr:+ crimson=0 2024-09-16T08:47:28.477 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' '' = --crimson ']' 2024-09-16T08:47:28.477 INFO:tasks.workunit.client.0.smithi160.stderr:+ color= 2024-09-16T08:47:28.478 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -t 1 ']' 2024-09-16T08:47:28.478 INFO:tasks.workunit.client.0.smithi160.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-09-16T08:47:28.478 INFO:tasks.workunit.client.0.smithi160.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-09-16T08:47:28.478 INFO:tasks.workunit.client.0.smithi160.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-09-16T08:47:28.478 INFO:tasks.workunit.client.0.smithi160.stderr:+ declare -A pids 2024-09-16T08:47:28.478 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.478 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.478 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_aio 2024-09-16T08:47:28.478 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_aio' 2024-09-16T08:47:28.479 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_aio 2024-09-16T08:47:28.479 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.479 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_aio 2024-09-16T08:47:28.479 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75671 2024-09-16T08:47:28.479 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_aio on pid 75671' 2024-09-16T08:47:28.480 INFO:tasks.workunit.client.0.smithi160.stdout:test api_aio on pid 75671 2024-09-16T08:47:28.482 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=75671 2024-09-16T08:47:28.483 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.483 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.483 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-09-16T08:47:28.483 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_aio_pp 2024-09-16T08:47:28.483 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_aio_pp' 2024-09-16T08:47:28.483 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_aio_pp 2024-09-16T08:47:28.483 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.483 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.483 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.483 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.483 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.483 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.483 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.483 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.484 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.484 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.484 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.484 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.484 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.484 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.484 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.484 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.484 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.484 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.484 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.484 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.485 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.485 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.485 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.485 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.485 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.485 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.485 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_aio_pp 2024-09-16T08:47:28.485 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75678 2024-09-16T08:47:28.485 INFO:tasks.workunit.client.0.smithi160.stdout:test api_aio_pp on pid 75678 2024-09-16T08:47:28.489 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_aio_pp on pid 75678' 2024-09-16T08:47:28.489 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=75678 2024-09-16T08:47:28.489 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.489 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.489 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-09-16T08:47:28.489 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_io 2024-09-16T08:47:28.489 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_io' 2024-09-16T08:47:28.489 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_io 2024-09-16T08:47:28.489 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.490 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.490 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.490 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.490 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.490 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.490 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.490 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.490 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.490 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.490 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.490 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.490 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.490 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.490 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.491 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.491 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.491 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.491 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.491 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.491 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.491 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.491 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.491 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.491 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.491 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.491 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.491 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.491 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.492 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.492 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.492 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.492 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.492 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.492 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.492 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.492 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.492 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.492 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.492 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.492 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.492 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.492 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.493 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.493 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.493 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.493 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_io 2024-09-16T08:47:28.493 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75687 2024-09-16T08:47:28.493 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_io on pid 75687' 2024-09-16T08:47:28.493 INFO:tasks.workunit.client.0.smithi160.stdout:test api_io on pid 75687 2024-09-16T08:47:28.497 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=75687 2024-09-16T08:47:28.497 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.497 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-09-16T08:47:28.497 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.497 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_io_pp 2024-09-16T08:47:28.497 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_io_pp' 2024-09-16T08:47:28.497 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_io_pp 2024-09-16T08:47:28.497 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.497 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.498 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.498 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.498 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.498 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.498 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.498 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.498 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.498 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.498 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.498 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.498 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.498 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.498 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.499 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.499 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.499 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.499 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.499 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.499 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.499 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.499 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.499 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.499 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.499 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.499 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.499 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.499 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.500 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.500 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.500 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.500 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.500 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.500 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.500 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.500 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.500 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.500 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.500 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.500 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.500 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.501 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.501 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.501 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.501 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.501 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.501 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.501 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.501 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.501 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.501 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.501 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.501 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.501 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.502 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_io_pp 2024-09-16T08:47:28.502 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75698 2024-09-16T08:47:28.502 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_io_pp on pid 75698' 2024-09-16T08:47:28.502 INFO:tasks.workunit.client.0.smithi160.stdout:test api_io_pp on pid 75698 2024-09-16T08:47:28.507 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=75698 2024-09-16T08:47:28.507 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.508 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-09-16T08:47:28.508 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.508 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_asio 2024-09-16T08:47:28.508 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_asio' 2024-09-16T08:47:28.508 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_asio 2024-09-16T08:47:28.508 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.508 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.508 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.508 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.508 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.508 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.508 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.509 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.509 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.509 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.509 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.509 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.509 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.509 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.509 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.509 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.509 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.509 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.509 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.509 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.509 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.510 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.510 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.510 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.510 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.510 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.510 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.510 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.510 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.510 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.510 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.510 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.510 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.510 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.511 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.511 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.511 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.511 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.511 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.511 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.511 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.511 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.511 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.511 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.511 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.511 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.511 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.511 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.512 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.512 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.512 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.512 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.512 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.512 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.512 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.512 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.512 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.512 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.512 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.512 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.512 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.513 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.513 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.513 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.513 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.513 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.513 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.513 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.513 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.513 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.513 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.513 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.513 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.513 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_asio 2024-09-16T08:47:28.513 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75712 2024-09-16T08:47:28.514 INFO:tasks.workunit.client.0.smithi160.stdout:test api_asio on pid 75712 2024-09-16T08:47:28.521 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_asio on pid 75712' 2024-09-16T08:47:28.521 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=75712 2024-09-16T08:47:28.522 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.522 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.522 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-09-16T08:47:28.522 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_list 2024-09-16T08:47:28.522 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.522 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.522 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_list' 2024-09-16T08:47:28.522 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.522 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.522 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.522 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_list 2024-09-16T08:47:28.522 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.522 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.523 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.523 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.523 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.523 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.523 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.523 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.523 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.523 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.523 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.523 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.523 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.523 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.523 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.523 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.524 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.524 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.524 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.524 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.524 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.524 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.524 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.524 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.524 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.524 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.524 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.524 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.524 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.524 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.525 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.525 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.525 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.525 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.525 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.525 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.525 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.525 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.525 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.525 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.525 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.525 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.525 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.526 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.526 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.526 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.526 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.526 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.526 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.526 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.526 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.526 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.526 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.526 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.526 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.526 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.526 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.527 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.527 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.527 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.527 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.527 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.527 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.527 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.527 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.527 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.527 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.527 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.527 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.527 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.527 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.528 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.528 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.528 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.528 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.528 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.528 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.528 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.528 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.528 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.528 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.528 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.528 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.528 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.529 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.529 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.529 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.529 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.529 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.529 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.529 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.529 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.529 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.529 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.529 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.529 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_list 2024-09-16T08:47:28.529 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.529 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.530 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75730 2024-09-16T08:47:28.530 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_list on pid 75730' 2024-09-16T08:47:28.530 INFO:tasks.workunit.client.0.smithi160.stdout:test api_list on pid 75730 2024-09-16T08:47:28.536 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=75730 2024-09-16T08:47:28.536 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.536 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.536 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.536 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/75671/exe 2024-09-16T08:47:28.536 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.536 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.536 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-09-16T08:47:28.536 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_lock 2024-09-16T08:47:28.536 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.537 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_lock' 2024-09-16T08:47:28.537 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.537 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_lock 2024-09-16T08:47:28.537 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.537 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.537 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.537 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.537 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.537 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.537 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.537 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.537 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.537 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.538 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.538 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.538 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.538 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.538 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.538 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.538 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.538 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.538 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.538 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.538 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.538 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.538 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.538 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.539 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.539 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.539 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.539 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.539 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.539 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.539 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.539 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.539 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.539 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.539 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.539 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.539 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.539 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.540 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.540 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.540 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.540 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.540 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.540 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.540 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.540 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.540 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.540 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.540 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.540 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.541 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.541 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.541 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.541 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.541 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.541 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.541 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.541 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.541 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.541 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.541 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.541 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.541 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.541 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.542 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.542 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.542 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.542 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.542 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.542 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.542 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.542 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.542 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.542 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.542 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.542 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.542 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.543 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.543 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.543 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.543 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.543 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.543 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.543 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.543 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.543 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.543 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.543 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.543 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.543 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.543 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.544 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.544 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.544 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.544 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.544 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.544 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.544 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.544 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.544 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.544 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.544 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.544 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.544 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.545 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2024-09-16T08:47:28.545 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_aio.log 2024-09-16T08:47:28.545 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_aio: /' 2024-09-16T08:47:28.545 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/75678/exe 2024-09-16T08:47:28.545 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.545 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.545 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.545 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_lock 2024-09-16T08:47:28.545 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.545 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.545 INFO:tasks.workunit.client.0.smithi160.stdout:test api_lock on pid 75749 2024-09-16T08:47:28.550 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75749 2024-09-16T08:47:28.551 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_lock on pid 75749' 2024-09-16T08:47:28.551 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=75749 2024-09-16T08:47:28.551 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.551 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.551 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_lock_pp 2024-09-16T08:47:28.551 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.551 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-09-16T08:47:28.551 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.551 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_lock_pp' 2024-09-16T08:47:28.551 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.551 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.551 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.552 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.552 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.552 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.552 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.552 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.552 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.552 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.552 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.552 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.552 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.552 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.552 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.553 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.553 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.553 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.553 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.553 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.553 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.553 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.553 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.553 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.553 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.553 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.553 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.553 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.554 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.554 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2024-09-16T08:47:28.554 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.554 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.554 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.554 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.554 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-09-16T08:47:28.554 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.554 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.554 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.554 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.554 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.554 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.554 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.555 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.555 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.555 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_aio_pp: /' 2024-09-16T08:47:28.555 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.555 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.555 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.555 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.555 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.555 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.555 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.555 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.555 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.555 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.555 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.555 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.556 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.556 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.556 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.556 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.556 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.556 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.556 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.556 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.556 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.556 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.556 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.556 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.557 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.557 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.557 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.557 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.557 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.557 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.557 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.557 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.557 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.557 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.557 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.557 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.557 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.557 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.558 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.558 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.558 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.558 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.558 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.558 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.558 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.558 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.558 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.558 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.558 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_lock_pp 2024-09-16T08:47:28.558 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.558 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.559 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.559 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.559 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.559 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.559 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.559 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.559 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.559 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.559 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.559 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.559 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.559 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.559 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.559 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.560 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.560 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.560 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.560 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.560 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/75687/exe 2024-09-16T08:47:28.560 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_lock_pp 2024-09-16T08:47:28.560 INFO:tasks.workunit.client.0.smithi160.stdout:test api_lock_pp on pid 75772 2024-09-16T08:47:28.562 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75772 2024-09-16T08:47:28.562 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_lock_pp on pid 75772' 2024-09-16T08:47:28.562 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=75772 2024-09-16T08:47:28.563 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.563 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.563 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_misc 2024-09-16T08:47:28.563 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_misc' 2024-09-16T08:47:28.563 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-09-16T08:47:28.563 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.563 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.563 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.563 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.563 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.563 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.563 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.563 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.563 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.564 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.564 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.564 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.564 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.564 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.564 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.564 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.564 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.564 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.564 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.564 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.564 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.564 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.565 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.565 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.565 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.565 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.565 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.565 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.565 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.565 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.565 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.565 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.566 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.566 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.566 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.566 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.566 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.566 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.566 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.566 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.566 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.566 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.566 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.566 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.566 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.567 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.567 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.567 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.567 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.567 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.567 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.567 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.567 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.567 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.567 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.567 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.567 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.567 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.567 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.568 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.568 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.568 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.568 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.568 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.568 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.568 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.568 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_misc 2024-09-16T08:47:28.568 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.568 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.568 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.568 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.568 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.570 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/75698/exe 2024-09-16T08:47:28.570 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.570 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.570 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.570 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.570 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.570 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2024-09-16T08:47:28.570 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.570 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.570 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.571 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.571 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.571 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.571 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.571 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.571 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.571 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.571 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.571 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.571 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.571 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.572 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.572 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.572 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.572 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.572 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.572 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.572 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.572 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.572 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.572 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.572 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.572 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.572 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.573 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.573 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.573 INFO:tasks.workunit.client.0.smithi160.stdout:test api_misc on pid 75790 2024-09-16T08:47:28.575 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.575 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.575 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.575 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.575 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.575 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.576 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.576 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.576 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_misc 2024-09-16T08:47:28.576 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75790 2024-09-16T08:47:28.576 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_misc on pid 75790' 2024-09-16T08:47:28.576 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=75790 2024-09-16T08:47:28.576 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.576 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.576 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.576 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.576 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.576 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.577 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.577 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.577 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.577 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.577 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.577 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.577 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.577 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.577 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_misc_pp 2024-09-16T08:47:28.577 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.577 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.577 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.578 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.578 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.578 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_io.log 2024-09-16T08:47:28.578 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.578 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_io_pp: /' 2024-09-16T08:47:28.578 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_misc_pp' 2024-09-16T08:47:28.578 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.578 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.578 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.578 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.578 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.578 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.579 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.579 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.579 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.579 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.579 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.579 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-09-16T08:47:28.579 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.579 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-09-16T08:47:28.579 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.579 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2024-09-16T08:47:28.579 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/75712/exe 2024-09-16T08:47:28.579 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.579 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.579 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.580 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.580 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.580 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.580 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.580 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.580 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.580 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.580 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.580 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.580 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.580 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.580 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.580 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.581 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.581 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.581 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.581 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.581 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.581 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.581 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.581 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.581 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.581 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.581 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_misc_pp 2024-09-16T08:47:28.581 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.581 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.581 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.582 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.582 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.582 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.582 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.582 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.582 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.582 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.582 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.582 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.582 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.582 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.582 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.582 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.583 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.583 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.583 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.583 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.583 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.583 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.583 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.583 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.583 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.583 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.583 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.583 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.583 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.583 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.583 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.584 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.584 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.584 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.584 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.584 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.584 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.584 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.584 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.584 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.584 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.584 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.584 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.584 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.584 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.585 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.585 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.585 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.585 INFO:tasks.workunit.client.0.smithi160.stdout:test api_misc_pp on pid 75812 2024-09-16T08:47:28.591 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_misc_pp 2024-09-16T08:47:28.591 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_asio.log 2024-09-16T08:47:28.591 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75812 2024-09-16T08:47:28.591 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_misc_pp on pid 75812' 2024-09-16T08:47:28.591 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=75812 2024-09-16T08:47:28.591 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.591 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.592 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.592 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.592 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.592 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.592 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.592 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.592 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.592 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.592 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.592 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2024-09-16T08:47:28.592 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.592 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.592 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.593 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.593 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.593 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_asio: /' 2024-09-16T08:47:28.593 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_tier_pp 2024-09-16T08:47:28.593 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_tier_pp' 2024-09-16T08:47:28.593 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.593 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.593 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.593 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.593 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.593 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.593 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.593 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.594 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.594 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.594 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.594 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.594 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.594 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.594 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.594 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.594 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.594 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.594 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.594 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.594 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.595 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.595 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.595 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.595 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.595 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.595 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.595 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.595 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.595 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.595 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.595 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.595 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.595 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.595 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.596 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.596 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.596 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.596 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.596 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.596 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.596 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.596 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.596 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.596 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.596 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.596 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.596 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.597 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.597 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.597 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.597 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.597 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.597 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.597 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.597 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.597 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.597 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.597 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.597 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.597 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.597 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.598 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.598 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.598 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.598 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.598 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.598 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.598 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/75749/exe 2024-09-16T08:47:28.598 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_io: /' 2024-09-16T08:47:28.598 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.598 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.598 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.598 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.598 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.598 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.599 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.599 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.599 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.599 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.599 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.599 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.599 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.599 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.599 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.599 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.599 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.599 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.599 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.600 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.600 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.600 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.600 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.600 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.600 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.600 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.600 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_lock: /' 2024-09-16T08:47:28.600 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2024-09-16T08:47:28.600 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.600 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_lock.log 2024-09-16T08:47:28.600 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.600 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.600 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.601 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.601 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.601 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.601 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.601 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.601 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.601 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.601 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.601 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.601 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.601 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.601 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.601 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.601 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_tier_pp 2024-09-16T08:47:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-09-16T08:47:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.604 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.604 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.604 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.604 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.604 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.604 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.604 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.604 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.604 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.604 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.604 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.604 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.604 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.605 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.605 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.605 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.605 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.605 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.605 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.605 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.605 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.605 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.605 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.605 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.605 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.605 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.605 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/75772/exe 2024-09-16T08:47:28.606 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_tier_pp 2024-09-16T08:47:28.606 INFO:tasks.workunit.client.0.smithi160.stdout:test api_tier_pp on pid 75843 2024-09-16T08:47:28.608 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:28 smithi169 ceph-mon[33465]: pgmap v47: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:47:28.609 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75843 2024-09-16T08:47:28.609 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_tier_pp on pid 75843' 2024-09-16T08:47:28.610 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=75843 2024-09-16T08:47:28.610 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.610 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.610 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-09-16T08:47:28.610 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.610 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_pool 2024-09-16T08:47:28.610 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_pool' 2024-09-16T08:47:28.610 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.610 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.610 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.610 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.610 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.610 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.611 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.611 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.611 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.611 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.611 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.611 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.611 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.611 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.611 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.611 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.611 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.611 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.611 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.612 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.612 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.612 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.612 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.612 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.612 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.612 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.612 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.612 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.612 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.612 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.612 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.612 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.613 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.613 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.613 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.613 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.613 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.613 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2024-09-16T08:47:28.613 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.613 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.613 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.613 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.613 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.613 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.613 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.614 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.614 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.614 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.614 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.614 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.614 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.614 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.614 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.614 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.614 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.614 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.614 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.614 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.615 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.615 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_lock_pp: /' 2024-09-16T08:47:28.615 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/75730/exe 2024-09-16T08:47:28.615 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-09-16T08:47:28.615 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.615 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_pool 2024-09-16T08:47:28.615 INFO:tasks.workunit.client.0.smithi160.stdout:test api_pool on pid 75861 2024-09-16T08:47:28.618 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_pool 2024-09-16T08:47:28.618 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75861 2024-09-16T08:47:28.618 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_pool on pid 75861' 2024-09-16T08:47:28.618 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=75861 2024-09-16T08:47:28.619 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.619 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.619 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.619 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.619 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.619 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.619 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.619 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.619 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.619 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.619 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.619 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.619 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2024-09-16T08:47:28.619 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_list.log 2024-09-16T08:47:28.620 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.620 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.620 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.620 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.620 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.620 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.620 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.620 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.620 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.620 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.620 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.620 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.620 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.621 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.621 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.621 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.621 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.621 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.621 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.621 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.621 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.621 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.621 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.621 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.621 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.621 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_list: /' 2024-09-16T08:47:28.621 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-09-16T08:47:28.622 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_snapshots 2024-09-16T08:47:28.622 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_snapshots' 2024-09-16T08:47:28.622 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.622 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.622 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.622 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.622 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.622 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.622 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.622 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.622 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.622 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.622 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.622 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.623 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.623 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.623 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.623 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.623 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.623 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.623 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.623 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.623 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.623 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.623 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.623 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.623 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.624 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.624 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.624 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.624 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.624 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.624 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.624 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.624 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.624 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.624 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.624 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.624 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_snapshots 2024-09-16T08:47:28.624 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.624 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.625 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.625 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.625 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.625 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.625 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.625 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.625 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.625 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.625 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.625 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.625 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.625 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.625 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.626 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.626 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.626 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.626 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.626 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.626 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.626 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.626 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.626 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.626 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.626 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.626 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.626 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.626 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.627 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.627 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.627 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.627 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.627 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/75790/exe 2024-09-16T08:47:28.627 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.627 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.627 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.627 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.627 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.627 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.627 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_snapshots 2024-09-16T08:47:28.627 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.627 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.628 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.628 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.628 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.628 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.628 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.628 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.628 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.628 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.628 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.628 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.628 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.628 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.628 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.629 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.629 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.629 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.629 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.629 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.629 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.629 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.629 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.629 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.629 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.629 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.629 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.629 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.630 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.630 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.630 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2024-09-16T08:47:28.630 INFO:tasks.workunit.client.0.smithi160.stdout:test api_snapshots on pid 75915 2024-09-16T08:47:28.632 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75915 2024-09-16T08:47:28.632 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_snapshots on pid 75915' 2024-09-16T08:47:28.632 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=75915 2024-09-16T08:47:28.632 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.632 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.632 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-09-16T08:47:28.632 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.632 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.632 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.633 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.633 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.633 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.633 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.633 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.633 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.633 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.633 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.633 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.633 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.633 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.633 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.633 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.633 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.634 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.634 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.634 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.634 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.634 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.634 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.634 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.634 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.634 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.634 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.634 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.634 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.634 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.635 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.635 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_misc.log 2024-09-16T08:47:28.635 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_snapshots_pp 2024-09-16T08:47:28.635 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_snapshots_pp' 2024-09-16T08:47:28.635 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.635 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.635 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.635 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.635 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.635 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.635 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.635 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.635 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.635 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.636 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.636 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.636 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.636 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.636 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.636 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.636 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.636 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.636 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.636 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.636 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.636 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.636 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.636 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.637 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.637 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.637 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.637 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.637 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.637 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.637 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.637 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.637 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.637 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.637 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.637 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.637 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.637 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.638 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.638 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.638 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_snapshots_pp 2024-09-16T08:47:28.638 INFO:tasks.workunit.client.0.smithi160.stdout:test api_snapshots_pp on pid 75952 2024-09-16T08:47:28.643 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_snapshots_pp 2024-09-16T08:47:28.643 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75952 2024-09-16T08:47:28.643 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_snapshots_pp on pid 75952' 2024-09-16T08:47:28.643 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=75952 2024-09-16T08:47:28.643 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.643 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.643 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.643 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.643 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.644 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.644 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.644 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.644 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.644 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.644 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.644 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.644 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_misc: /' 2024-09-16T08:47:28.644 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.644 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.644 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.644 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.645 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.645 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.645 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.645 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.645 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.645 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.645 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.645 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.645 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.645 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.645 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.645 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.645 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.646 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.646 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.646 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.646 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.646 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.646 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.646 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.646 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.646 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.646 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.646 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.646 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.646 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.646 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-09-16T08:47:28.647 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.647 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.647 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.647 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.647 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.647 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.647 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.647 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.647 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.647 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.647 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.648 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.648 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.648 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.648 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.648 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.648 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.648 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.648 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.648 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.648 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.648 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.648 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.648 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.649 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.649 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.649 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.649 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/75812/exe 2024-09-16T08:47:28.649 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.649 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.649 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.649 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.649 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.649 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.649 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.649 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.649 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.649 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.650 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.650 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.650 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.650 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.650 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.650 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.650 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.650 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.650 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.650 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.650 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.650 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.650 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.651 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.651 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.651 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.651 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.651 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.651 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.651 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.651 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.651 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.651 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.651 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.651 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.651 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.651 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.652 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.652 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.652 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.652 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.652 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.652 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.652 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.652 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.652 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.652 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_stat 2024-09-16T08:47:28.652 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.652 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.652 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_stat' 2024-09-16T08:47:28.653 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.653 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.653 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.653 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.653 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.653 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.653 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_misc_pp: /' 2024-09-16T08:47:28.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2024-09-16T08:47:28.654 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.654 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.654 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.654 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.654 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.654 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.654 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.654 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.654 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.654 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.654 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.654 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.655 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.655 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.655 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.655 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.655 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.655 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.655 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.655 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.655 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.655 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.655 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_stat 2024-09-16T08:47:28.655 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.655 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.656 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.656 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.656 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.656 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.656 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.656 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-09-16T08:47:28.656 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.656 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.656 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.656 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.656 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.656 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.656 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.656 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.657 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.657 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.657 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.657 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.657 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.657 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.657 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.657 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.657 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.657 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.657 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.657 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.658 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.658 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.658 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.658 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.658 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.658 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.658 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.658 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.658 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.658 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.658 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.658 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/75861/exe 2024-09-16T08:47:28.658 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.658 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.659 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.659 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.659 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.659 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.659 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.659 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.659 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.659 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.659 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.659 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.659 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.659 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.659 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.660 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.660 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.660 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.660 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.660 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.660 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.660 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.660 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.660 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.660 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.661 INFO:tasks.workunit.client.0.smithi160.stdout:test api_stat on pid 76027 2024-09-16T08:47:28.662 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_stat 2024-09-16T08:47:28.662 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.662 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.662 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76027 2024-09-16T08:47:28.663 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_stat on pid 76027' 2024-09-16T08:47:28.663 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.663 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76027 2024-09-16T08:47:28.663 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.663 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.663 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.663 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_stat_pp 2024-09-16T08:47:28.663 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2024-09-16T08:47:28.663 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.663 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.663 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.663 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.663 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.663 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_stat_pp' 2024-09-16T08:47:28.664 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.664 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.664 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.664 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.664 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.664 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.664 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.664 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.664 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_stat_pp 2024-09-16T08:47:28.664 INFO:tasks.workunit.client.0.smithi160.stdout:test api_stat_pp on pid 76044 2024-09-16T08:47:28.674 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_stat_pp 2024-09-16T08:47:28.674 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76044 2024-09-16T08:47:28.674 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_stat_pp on pid 76044' 2024-09-16T08:47:28.674 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76044 2024-09-16T08:47:28.674 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.674 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.674 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.674 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.674 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.674 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.674 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.675 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.675 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.675 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.675 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.675 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.675 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.675 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.675 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.675 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.675 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.675 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.675 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.675 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.676 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.676 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.676 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.676 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.676 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.676 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.676 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.676 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.676 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.676 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.676 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.676 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.676 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.677 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-09-16T08:47:28.677 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.677 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.677 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.677 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.677 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.677 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.677 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.677 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.677 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.677 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.677 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-09-16T08:47:28.677 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.677 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.678 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.678 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.678 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.678 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.678 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.678 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.678 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.678 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.678 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_watch_notify 2024-09-16T08:47:28.678 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/75915/exe 2024-09-16T08:47:28.678 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_watch_notify' 2024-09-16T08:47:28.678 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.678 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.679 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.679 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_pool.log 2024-09-16T08:47:28.679 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.679 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.679 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.679 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.679 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.679 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.679 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.679 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.679 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.679 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.679 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.679 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.680 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.680 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.680 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.680 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.680 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.680 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.680 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.680 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.680 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.680 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.680 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.680 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.680 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.680 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.681 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.681 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.681 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.681 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.681 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.681 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.681 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.681 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.681 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.681 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.681 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.681 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.681 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.682 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.682 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.682 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.682 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.682 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.682 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.682 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.682 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.682 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.682 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_watch_notify 2024-09-16T08:47:28.682 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.682 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.682 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.683 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.683 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.683 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.683 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.683 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.683 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.683 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2024-09-16T08:47:28.683 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.683 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-09-16T08:47:28.683 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_snapshots: /' 2024-09-16T08:47:28.683 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.683 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.683 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.683 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.684 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.684 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.684 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.684 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.684 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.684 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.684 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.684 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.684 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.684 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.684 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.684 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.684 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.684 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.685 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.685 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.685 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.685 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.685 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.685 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.685 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.685 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.685 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.685 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.685 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.685 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.685 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.686 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.686 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.686 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.686 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.686 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.686 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.686 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.686 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_pool: /' 2024-09-16T08:47:28.686 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.686 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.686 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.686 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.686 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.686 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.686 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/75843/exe 2024-09-16T08:47:28.687 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.687 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.687 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.687 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.687 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.687 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.687 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.687 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.687 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.687 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.687 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.687 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.687 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.688 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.688 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.688 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.688 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.688 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.688 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.688 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.688 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.688 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.688 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.688 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.688 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.688 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.689 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.689 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.689 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.689 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.689 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.689 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.689 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.689 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.689 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.689 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.689 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.689 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.689 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.689 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.689 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.690 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.690 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.690 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.690 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.690 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.690 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.690 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.690 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.690 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.690 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.690 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.690 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.690 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.691 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.691 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.691 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.691 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.691 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.691 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.691 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.691 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.691 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.691 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.691 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.691 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.691 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.691 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.692 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.692 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.692 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.692 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-09-16T08:47:28.692 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.692 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.692 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.692 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.692 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.692 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.692 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.692 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.693 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.693 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.693 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.693 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.693 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.693 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.693 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.693 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.693 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.693 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_watch_notify 2024-09-16T08:47:28.693 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76095 2024-09-16T08:47:28.693 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_watch_notify on pid 76095' 2024-09-16T08:47:28.694 INFO:tasks.workunit.client.0.smithi160.stdout:test api_watch_notify on pid 76095 2024-09-16T08:47:28.697 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76095 2024-09-16T08:47:28.697 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.697 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.697 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.697 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-09-16T08:47:28.697 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_tier_pp: /' 2024-09-16T08:47:28.698 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_watch_notify_pp 2024-09-16T08:47:28.698 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_watch_notify_pp' 2024-09-16T08:47:28.698 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_watch_notify_pp 2024-09-16T08:47:28.698 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.698 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.698 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.698 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.698 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.698 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.698 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.698 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2024-09-16T08:47:28.698 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.698 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.699 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.699 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.699 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.699 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.699 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.699 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.699 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.699 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.699 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.699 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.699 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.699 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.699 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.699 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.700 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.700 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.700 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.700 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.700 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_watch_notify_pp 2024-09-16T08:47:28.700 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.700 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.700 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76116 2024-09-16T08:47:28.700 INFO:tasks.workunit.client.0.smithi160.stdout:test api_watch_notify_pp on pid 76116 2024-09-16T08:47:28.703 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_watch_notify_pp on pid 76116' 2024-09-16T08:47:28.703 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76116 2024-09-16T08:47:28.703 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.703 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.704 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.704 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.704 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.704 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.704 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.704 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.704 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.704 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.704 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.704 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.704 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.704 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.704 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.705 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.705 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.705 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.705 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.705 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.705 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.705 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.705 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.705 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.705 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.705 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.705 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_cmd 2024-09-16T08:47:28.705 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_cmd' 2024-09-16T08:47:28.706 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/75952/exe 2024-09-16T08:47:28.706 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.706 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-09-16T08:47:28.706 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.706 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.706 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.706 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.706 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.706 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.706 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.706 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76044/exe 2024-09-16T08:47:28.706 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.706 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.706 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.707 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2024-09-16T08:47:28.707 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.707 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.707 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_cmd 2024-09-16T08:47:28.707 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.707 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.707 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.707 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.707 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.707 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.707 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.707 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.707 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.707 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.708 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.708 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.708 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.708 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.708 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.708 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-09-16T08:47:28.708 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_stat_pp: /' 2024-09-16T08:47:28.708 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.708 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-09-16T08:47:28.708 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.708 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.708 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.708 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.709 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.709 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.709 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.709 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.709 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.709 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.709 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.709 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.709 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.709 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.709 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.709 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.709 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.709 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.710 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.710 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.710 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2024-09-16T08:47:28.710 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_cmd 2024-09-16T08:47:28.710 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76147 2024-09-16T08:47:28.710 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_cmd on pid 76147' 2024-09-16T08:47:28.710 INFO:tasks.workunit.client.0.smithi160.stdout:test api_cmd on pid 76147 2024-09-16T08:47:28.713 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76147 2024-09-16T08:47:28.713 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.714 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.714 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_cmd_pp 2024-09-16T08:47:28.714 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_cmd_pp' 2024-09-16T08:47:28.714 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_cmd_pp 2024-09-16T08:47:28.714 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-09-16T08:47:28.714 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.714 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.714 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.714 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.714 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.714 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.714 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.714 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.715 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.715 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.715 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.715 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.715 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.715 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.715 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.715 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.715 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.715 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.715 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.715 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.715 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.715 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.716 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.716 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-09-16T08:47:28.716 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.716 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.716 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.716 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.716 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.716 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.716 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.716 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.716 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.716 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.716 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.717 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.717 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_cmd_pp 2024-09-16T08:47:28.717 INFO:tasks.workunit.client.0.smithi160.stdout:test api_cmd_pp on pid 76164 2024-09-16T08:47:28.721 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76164 2024-09-16T08:47:28.722 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_cmd_pp on pid 76164' 2024-09-16T08:47:28.722 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76164 2024-09-16T08:47:28.722 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.722 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.722 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-09-16T08:47:28.722 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_service 2024-09-16T08:47:28.722 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_service' 2024-09-16T08:47:28.722 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.722 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.722 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.722 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.722 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.723 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.723 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.723 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.723 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.723 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.723 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.723 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.723 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.723 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.723 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.723 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.723 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.723 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.723 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.724 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.724 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.724 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.724 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.724 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.724 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.724 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.724 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.724 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.724 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.724 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.724 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.724 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.724 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.725 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.725 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.725 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.725 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.725 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.725 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.725 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.725 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.725 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.725 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.725 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.725 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.725 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.725 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.726 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.726 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.726 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.726 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.726 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.726 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.726 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.726 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.726 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.726 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.726 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.726 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.726 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.727 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.727 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.727 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.727 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.727 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.727 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.727 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.727 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.727 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.727 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.727 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.727 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.727 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.728 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.728 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.728 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.728 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.728 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.728 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.728 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.728 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.728 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.728 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.728 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.728 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.728 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.729 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.729 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.729 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.729 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.729 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.729 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.729 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.729 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.729 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.729 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.729 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.729 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.729 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.730 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.730 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.730 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.730 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.730 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.730 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_service 2024-09-16T08:47:28.730 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.730 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.730 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.730 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.730 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.730 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.730 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.731 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.731 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.731 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.731 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.731 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.731 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.731 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.731 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.731 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.731 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.731 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.731 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.731 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.732 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.732 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.732 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.732 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.732 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.732 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.732 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.732 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.732 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.732 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.732 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.732 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.732 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.733 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.733 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.733 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.733 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.733 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.733 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.733 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.733 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.733 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.733 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.733 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.733 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.733 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.734 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.734 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.734 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.734 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.734 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.734 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.734 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.734 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.734 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.734 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.734 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.734 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.734 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.735 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.735 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.735 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.735 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.735 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.735 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.735 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.735 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.735 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.735 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.735 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.735 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.735 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.736 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.736 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.736 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.736 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.736 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.736 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.736 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.736 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.736 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.736 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.736 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.736 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.737 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.737 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.737 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.737 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.737 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.737 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.737 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.737 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.737 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.737 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.737 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.737 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.737 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.738 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.738 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.738 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.738 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.738 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.738 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.738 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.738 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.738 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_service 2024-09-16T08:47:28.738 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76245 2024-09-16T08:47:28.738 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_service on pid 76245' 2024-09-16T08:47:28.738 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76245 2024-09-16T08:47:28.738 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.738 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.739 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.739 INFO:tasks.workunit.client.0.smithi160.stdout:test api_service on pid 76245 2024-09-16T08:47:28.742 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-09-16T08:47:28.742 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76027/exe 2024-09-16T08:47:28.742 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_service_pp 2024-09-16T08:47:28.742 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_service_pp' 2024-09-16T08:47:28.742 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.742 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.742 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.742 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.742 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.742 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.742 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.742 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.743 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.743 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.743 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.743 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.743 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.743 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.743 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.743 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.743 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.743 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.743 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.743 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.743 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.744 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.744 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.744 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.744 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.744 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.744 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.744 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.744 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.744 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.744 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.744 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.744 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.744 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.744 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.745 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.745 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.745 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.745 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.745 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.745 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.745 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_stat.log 2024-09-16T08:47:28.745 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.745 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.745 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.745 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.745 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.745 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.746 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.746 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.746 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.746 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2024-09-16T08:47:28.746 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_stat: /' 2024-09-16T08:47:28.746 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_service_pp 2024-09-16T08:47:28.746 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.746 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.746 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.746 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.746 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.746 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.746 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.746 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.746 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.747 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.747 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.747 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.747 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.747 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.747 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.747 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.747 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.747 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.747 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.747 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.747 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.747 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.747 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.748 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.748 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.748 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.748 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.748 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.748 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.748 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.748 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.748 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.748 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.748 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.748 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.748 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.748 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.749 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.749 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.749 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.749 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.749 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.749 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.749 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.749 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.749 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.749 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.749 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.749 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_service_pp 2024-09-16T08:47:28.750 INFO:tasks.workunit.client.0.smithi160.stdout:test api_service_pp on pid 76270 2024-09-16T08:47:28.752 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.752 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.752 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.752 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.752 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.752 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.753 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76270 2024-09-16T08:47:28.753 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.753 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_service_pp on pid 76270' 2024-09-16T08:47:28.753 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76270 2024-09-16T08:47:28.753 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.753 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.753 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.753 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.753 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.753 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.753 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.753 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-09-16T08:47:28.754 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_c_write_operations 2024-09-16T08:47:28.754 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_c_write_operations' 2024-09-16T08:47:28.754 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.754 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.754 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.754 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.754 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.754 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.754 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.754 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.754 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.754 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.754 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.754 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.755 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.755 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.755 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.755 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_c_write_operations 2024-09-16T08:47:28.755 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.755 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.755 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.755 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.755 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.755 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.755 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.755 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.755 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.755 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.756 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.756 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.756 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.756 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.756 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.756 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.756 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.756 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76095/exe 2024-09-16T08:47:28.756 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.756 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.756 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.756 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.756 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.756 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.757 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.757 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.757 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.757 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.757 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.757 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.757 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.757 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.757 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.757 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.757 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.757 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.758 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.758 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.758 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76116/exe 2024-09-16T08:47:28.758 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.758 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.758 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.758 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.758 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.758 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.758 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.758 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.758 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.758 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.759 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.759 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.759 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.759 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.759 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.759 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.759 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.759 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.759 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.759 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.759 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.759 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.759 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.759 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.760 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.760 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.760 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.760 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.760 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.760 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.760 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.760 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.760 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.760 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.760 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.760 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.760 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.760 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.761 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.761 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.761 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.761 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.761 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.761 INFO:tasks.workunit.client.0.smithi160.stdout:test api_c_write_operations on pid 76303 2024-09-16T08:47:28.768 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_c_write_operations 2024-09-16T08:47:28.768 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.768 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.769 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.769 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.769 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.769 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.769 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.769 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76303 2024-09-16T08:47:28.769 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_c_write_operations on pid 76303' 2024-09-16T08:47:28.769 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76303 2024-09-16T08:47:28.769 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.769 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.769 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.769 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.769 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.770 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.770 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.770 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.770 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.770 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.770 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.770 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.770 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76147/exe 2024-09-16T08:47:28.770 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2024-09-16T08:47:28.770 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-09-16T08:47:28.770 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.771 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.771 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.771 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.771 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.771 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.771 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.771 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.771 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.771 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.771 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.771 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.771 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.771 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.772 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.772 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.772 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.772 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.772 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2024-09-16T08:47:28.772 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.772 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-09-16T08:47:28.772 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.772 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.772 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.772 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.772 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.773 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.773 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.773 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.773 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.773 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.773 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2024-09-16T08:47:28.773 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.773 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.773 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.773 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.773 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.773 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.773 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.774 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.774 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.774 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.774 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.774 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.774 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.774 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.774 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.774 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.774 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.774 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.774 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.774 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.775 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.775 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76245/exe 2024-09-16T08:47:28.775 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.775 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_cmd: /' 2024-09-16T08:47:28.775 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.775 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.775 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.775 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.775 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.775 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.775 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.775 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.775 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.775 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.776 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_service: /' 2024-09-16T08:47:28.776 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2024-09-16T08:47:28.776 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_service.log 2024-09-16T08:47:28.776 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-09-16T08:47:28.776 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-09-16T08:47:28.776 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.776 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_c_read_operations 2024-09-16T08:47:28.776 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.776 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.776 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.776 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.776 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.776 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.777 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.777 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_c_read_operations' 2024-09-16T08:47:28.777 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.777 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.777 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.777 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.777 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.777 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.777 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.777 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.777 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.777 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.777 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_cmd.log 2024-09-16T08:47:28.777 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.778 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.778 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.778 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.778 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.778 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.778 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.778 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.778 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.778 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.778 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.778 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.778 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.778 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.779 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.779 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.779 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.779 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.779 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.779 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.779 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.779 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.779 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.779 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.779 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.779 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.779 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.779 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_watch_notify: /' 2024-09-16T08:47:28.780 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.780 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.780 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.780 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.780 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.780 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.780 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.780 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.780 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.780 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.780 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.780 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.780 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.780 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.781 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.781 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.781 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.781 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.781 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.781 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.781 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.781 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.781 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.781 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_c_read_operations 2024-09-16T08:47:28.781 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.781 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.781 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.782 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.782 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.782 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.782 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.782 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.782 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.782 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.782 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.782 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.782 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.782 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.782 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.782 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.782 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.783 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.783 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.783 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.783 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.783 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.783 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.783 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.783 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.783 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.783 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.783 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.783 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.783 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.784 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.784 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.784 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.784 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.784 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.784 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.784 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.784 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.784 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.784 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.784 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.784 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.784 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.785 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.785 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.785 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.785 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.785 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.785 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.785 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.785 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.785 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.785 INFO:tasks.workunit.client.0.smithi160.stdout:test api_c_read_operations on pid 76370 2024-09-16T08:47:28.792 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_c_read_operations 2024-09-16T08:47:28.792 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76370 2024-09-16T08:47:28.792 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_c_read_operations on pid 76370' 2024-09-16T08:47:28.792 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76370 2024-09-16T08:47:28.792 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.792 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.792 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.792 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.792 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.793 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.793 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.793 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.793 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.793 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.793 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.793 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.793 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.793 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.793 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76164/exe 2024-09-16T08:47:28.793 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.794 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.794 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.794 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.794 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.794 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.794 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.794 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-09-16T08:47:28.794 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.794 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.794 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.794 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.794 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.794 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.794 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.795 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.795 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.795 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.795 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.795 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.795 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.795 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.795 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.795 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.795 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.795 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.795 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.795 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.795 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.796 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.796 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.796 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.796 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.796 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2024-09-16T08:47:28.796 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-09-16T08:47:28.796 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.796 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.796 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.796 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.796 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.796 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.796 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.797 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.797 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.797 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.797 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.797 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.797 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.797 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.797 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.797 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.797 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.797 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.797 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.797 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s list_parallel 2024-09-16T08:47:28.798 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' list_parallel' 2024-09-16T08:47:28.798 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.798 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.798 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.798 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.798 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.798 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.798 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-09-16T08:47:28.798 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.798 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.798 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.798 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.798 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.798 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.799 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.799 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.799 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.799 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.799 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.799 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.799 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.799 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.799 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.799 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.799 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.799 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.799 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.800 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.800 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.800 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.800 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.800 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.800 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.800 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76270/exe 2024-09-16T08:47:28.800 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.800 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.800 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.800 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.800 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.800 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.801 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.801 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.801 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.801 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.801 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.801 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.801 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.801 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.801 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.801 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.801 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.801 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.801 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.802 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.802 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.802 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.802 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.802 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.802 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.802 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.802 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.802 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.802 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.802 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.802 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.802 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.802 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.803 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.803 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.803 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.803 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.803 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.803 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.803 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.803 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.803 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.803 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.803 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.803 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.803 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.803 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo list_parallel 2024-09-16T08:47:28.804 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_service_pp: /' 2024-09-16T08:47:28.804 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=list_parallel 2024-09-16T08:47:28.804 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2024-09-16T08:47:28.804 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76426 2024-09-16T08:47:28.804 INFO:tasks.workunit.client.0.smithi160.stdout:test list_parallel on pid 76426 2024-09-16T08:47:28.807 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test list_parallel on pid 76426' 2024-09-16T08:47:28.807 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-09-16T08:47:28.807 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76426 2024-09-16T08:47:28.807 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.807 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.807 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-09-16T08:47:28.807 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.807 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.807 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.807 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.807 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.808 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.808 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.808 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.808 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.808 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.808 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.808 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.808 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.808 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.808 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.808 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s open_pools_parallel 2024-09-16T08:47:28.809 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' open_pools_parallel' 2024-09-16T08:47:28.809 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.809 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.809 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.809 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.809 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.809 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.809 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.809 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.809 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.809 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.809 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.810 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.810 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.810 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.810 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.810 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.810 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.810 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.810 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.812 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.812 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.812 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.812 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.812 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.812 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo open_pools_parallel 2024-09-16T08:47:28.812 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.812 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.812 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.812 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.812 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.812 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.812 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.812 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.813 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.813 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.813 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.813 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.813 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.813 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.813 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.813 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.813 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.813 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.813 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.813 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.814 INFO:tasks.workunit.client.0.smithi160.stdout:test open_pools_parallel on pid 76461 2024-09-16T08:47:28.816 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=open_pools_parallel 2024-09-16T08:47:28.816 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76461 2024-09-16T08:47:28.816 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test open_pools_parallel on pid 76461' 2024-09-16T08:47:28.817 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76461 2024-09-16T08:47:28.817 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-16T08:47:28.817 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.817 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.817 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.817 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.817 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.817 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.817 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.817 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.817 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.817 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.818 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.818 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.818 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.818 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.818 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.818 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.818 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.818 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.818 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.818 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.818 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.818 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.818 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.818 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.819 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-09-16T08:47:28.819 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.819 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.819 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s delete_pools_parallel 2024-09-16T08:47:28.819 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.819 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.819 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.819 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.819 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.819 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.819 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.819 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.819 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.819 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' delete_pools_parallel' 2024-09-16T08:47:28.820 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.820 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.820 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.820 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.820 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.820 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.820 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.820 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.820 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.820 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.820 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.820 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.820 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo delete_pools_parallel 2024-09-16T08:47:28.820 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.821 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.821 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.821 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.821 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.821 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.821 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.821 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.821 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.821 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.821 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.821 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.821 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.821 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.822 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.822 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.822 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.822 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.822 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.822 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.822 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.822 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.822 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.822 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.822 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.822 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.823 INFO:tasks.workunit.client.0.smithi160.stdout:test delete_pools_parallel on pid 76490 2024-09-16T08:47:28.824 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=delete_pools_parallel 2024-09-16T08:47:28.824 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76490 2024-09-16T08:47:28.824 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test delete_pools_parallel on pid 76490' 2024-09-16T08:47:28.825 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76490 2024-09-16T08:47:28.825 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-16T08:47:28.825 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.825 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s cls 2024-09-16T08:47:28.825 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.825 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' cls' 2024-09-16T08:47:28.825 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76303/exe 2024-09-16T08:47:28.825 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-09-16T08:47:28.825 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo cls 2024-09-16T08:47:28.825 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.825 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.825 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.826 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.826 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.826 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.826 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.826 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.826 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.826 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.826 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.826 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.826 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.826 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.826 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.826 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.826 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.827 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.827 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.827 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.827 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.827 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.827 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.827 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.827 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.827 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.827 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.827 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.827 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.827 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.827 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.828 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.828 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.828 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.828 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.828 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.828 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.828 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.828 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.828 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.828 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.828 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.828 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.828 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76370/exe 2024-09-16T08:47:28.829 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.829 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.829 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.829 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.829 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.829 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.829 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.829 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.829 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.829 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.829 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.829 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.829 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.829 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.830 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.830 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.830 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.830 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.830 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.830 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.830 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.830 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.830 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.830 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.830 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.830 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.830 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.831 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.831 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.831 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.831 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.831 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.831 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.831 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.831 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.831 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.831 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.831 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.831 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.831 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.831 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.832 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.832 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.832 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.832 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.832 INFO:tasks.workunit.client.0.smithi160.stdout:test cls on pid 76512 2024-09-16T08:47:28.836 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.836 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.836 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=cls 2024-09-16T08:47:28.836 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.836 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.836 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.836 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.836 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.837 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76512 2024-09-16T08:47:28.837 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test cls on pid 76512' 2024-09-16T08:47:28.837 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76512 2024-09-16T08:47:28.837 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-16T08:47:28.837 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.837 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.837 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.837 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.837 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2024-09-16T08:47:28.837 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-09-16T08:47:28.837 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2024-09-16T08:47:28.837 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.838 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.838 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.838 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.838 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s cmd 2024-09-16T08:47:28.838 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.838 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.838 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.838 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.838 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.838 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.838 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.838 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.838 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' cmd' 2024-09-16T08:47:28.839 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.839 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.839 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.839 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.839 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.839 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.839 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-09-16T08:47:28.839 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-09-16T08:47:28.839 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo cmd 2024-09-16T08:47:28.839 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.839 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.839 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.839 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.840 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.840 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.840 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.840 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.840 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.840 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.840 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.840 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.840 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.840 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.840 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.840 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.840 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.840 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.841 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.841 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.841 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.841 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.841 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.841 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.841 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.841 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.841 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.841 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.841 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.841 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.842 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.842 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.842 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.842 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.842 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.842 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.842 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.842 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.842 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-09-16T08:47:28.842 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.842 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.842 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.842 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.843 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.843 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.843 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.843 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.843 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.843 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-09-16T08:47:28.843 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.843 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.843 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.843 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.843 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.843 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.843 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.843 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.844 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.844 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.844 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.844 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.844 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.844 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.844 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.844 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.844 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.844 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.844 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.844 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.844 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.844 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.845 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.845 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.845 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.845 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.845 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.845 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.845 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.845 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.845 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.845 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.845 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.845 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.845 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.846 INFO:tasks.workunit.client.0.smithi160.stdout:test cmd on pid 76559 2024-09-16T08:47:28.848 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=cmd 2024-09-16T08:47:28.849 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76559 2024-09-16T08:47:28.849 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test cmd on pid 76559' 2024-09-16T08:47:28.849 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76559 2024-09-16T08:47:28.849 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76426/exe 2024-09-16T08:47:28.849 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-16T08:47:28.849 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.849 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s handler_error 2024-09-16T08:47:28.849 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' handler_error' 2024-09-16T08:47:28.849 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.849 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo handler_error 2024-09-16T08:47:28.849 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.849 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.850 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.850 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.850 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.850 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.850 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.850 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-09-16T08:47:28.850 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.850 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.850 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.850 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.850 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.850 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.850 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.851 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.851 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.851 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.851 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.851 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.851 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.851 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.851 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.851 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.851 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.851 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.851 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.851 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.851 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.852 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.852 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.852 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.852 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.852 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.852 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.852 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.852 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.852 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.852 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.852 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.852 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.852 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.852 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.853 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.853 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_list_parallel.log 2024-09-16T08:47:28.853 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.853 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.853 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.853 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.853 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.853 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.853 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.853 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.853 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.853 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.853 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.853 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.854 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.854 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.854 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.854 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.854 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.854 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.854 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.854 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.854 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2024-09-16T08:47:28.854 INFO:tasks.workunit.client.0.smithi160.stdout:test handler_error on pid 76574 2024-09-16T08:47:28.856 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=handler_error 2024-09-16T08:47:28.856 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76574 2024-09-16T08:47:28.856 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test handler_error on pid 76574' 2024-09-16T08:47:28.856 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76574 2024-09-16T08:47:28.856 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-16T08:47:28.856 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.856 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ list_parallel: /' 2024-09-16T08:47:28.856 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.857 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.857 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.857 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.857 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.857 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.857 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.857 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.857 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.857 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.857 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.857 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.857 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.857 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.857 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.858 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.858 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s io 2024-09-16T08:47:28.858 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.858 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' io' 2024-09-16T08:47:28.858 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-09-16T08:47:28.858 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo io 2024-09-16T08:47:28.858 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.858 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.858 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.858 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.858 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.858 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.858 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.858 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.859 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=io 2024-09-16T08:47:28.859 INFO:tasks.workunit.client.0.smithi160.stdout:test io on pid 76590 2024-09-16T08:47:28.867 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.867 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.867 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76590 2024-09-16T08:47:28.867 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.867 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test io on pid 76590' 2024-09-16T08:47:28.867 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76590 2024-09-16T08:47:28.867 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.867 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-16T08:47:28.868 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.868 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.868 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.868 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.868 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.868 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.868 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.868 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.868 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.868 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.868 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-09-16T08:47:28.868 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.869 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.869 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.869 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.869 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.869 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.869 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.869 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.869 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.869 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.869 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.869 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.869 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.869 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.870 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.870 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.870 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.870 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.870 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.870 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.870 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.870 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.870 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.870 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.870 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.870 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.870 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.870 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.871 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.871 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.871 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.871 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.871 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.871 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.871 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.871 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.871 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.871 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.871 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.871 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.871 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.871 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.872 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.872 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.872 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.872 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.872 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.872 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.872 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.872 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.872 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.872 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.872 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.872 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.872 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.873 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.873 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.873 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.873 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.873 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.873 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.873 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.873 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.873 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.873 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.873 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.873 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.873 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.873 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.874 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.874 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.874 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.874 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.874 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.874 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.874 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.874 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.874 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.874 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.874 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.874 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.874 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.874 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.875 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76461/exe 2024-09-16T08:47:28.875 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.875 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.875 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.875 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.875 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.875 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.875 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.875 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.875 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.875 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.875 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.875 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.875 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.876 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.876 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.876 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.876 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.876 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.876 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.876 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.876 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.876 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.876 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.876 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.876 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-09-16T08:47:28.876 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.877 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.877 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.877 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.877 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.877 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.877 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.877 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.877 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.877 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.877 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.877 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.877 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.877 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.877 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.878 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.878 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.878 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.878 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.878 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.878 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.878 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.878 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.878 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s ec_io 2024-09-16T08:47:28.878 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.878 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' ec_io' 2024-09-16T08:47:28.878 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.878 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2024-09-16T08:47:28.878 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-09-16T08:47:28.879 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.879 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.879 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.879 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.879 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.879 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.879 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.879 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.879 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.879 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.879 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.879 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.879 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.879 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.879 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.880 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.880 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.880 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.880 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.880 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.880 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.880 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.880 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.880 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.880 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.880 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.880 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.880 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo ec_io 2024-09-16T08:47:28.881 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.881 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.881 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.881 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.881 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.881 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.881 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.881 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.881 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.881 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.881 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.881 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.881 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.881 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.882 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.882 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.882 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.882 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.882 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.882 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.882 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.882 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.882 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76490/exe 2024-09-16T08:47:28.882 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.882 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.882 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.882 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.882 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.882 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.883 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.883 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.883 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.883 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.883 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.883 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.883 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.883 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.883 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.883 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.883 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.883 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.883 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.883 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.884 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=ec_io 2024-09-16T08:47:28.884 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76643 2024-09-16T08:47:28.884 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76574/exe 2024-09-16T08:47:28.884 INFO:tasks.workunit.client.0.smithi160.stdout:test ec_io on pid 76643 2024-09-16T08:47:28.888 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test ec_io on pid 76643' 2024-09-16T08:47:28.888 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76643 2024-09-16T08:47:28.888 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-16T08:47:28.889 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.889 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.889 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.889 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.889 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.889 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.889 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.889 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.889 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.889 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.889 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.889 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.889 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.890 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.890 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.890 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.890 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.890 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.890 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.890 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.890 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.890 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.890 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.890 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.890 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.890 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.891 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.891 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.891 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.891 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.891 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.891 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.891 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.891 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.891 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.891 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.891 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.891 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.891 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.891 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.892 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ handler_error: /' 2024-09-16T08:47:28.892 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_handler_error 2024-09-16T08:47:28.892 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s list 2024-09-16T08:47:28.892 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' list' 2024-09-16T08:47:28.892 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2024-09-16T08:47:28.892 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-09-16T08:47:28.892 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.892 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.892 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.892 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.892 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.892 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_handler_error.log 2024-09-16T08:47:28.893 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.893 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.893 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.893 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.893 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.893 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.893 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.893 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.893 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.893 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.893 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo list 2024-09-16T08:47:28.893 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.893 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.894 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.894 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.894 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.894 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.894 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.894 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.894 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.894 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.894 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.894 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.894 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.894 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.894 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.894 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.895 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.895 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.895 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.895 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.895 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.895 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.895 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.895 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.895 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.895 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.895 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.895 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.895 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.895 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.896 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.896 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.896 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.896 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.896 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.896 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.896 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.896 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.896 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.896 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-09-16T08:47:28.896 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-09-16T08:47:28.896 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.896 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.897 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.897 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.897 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.897 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.897 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.897 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.897 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.897 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.897 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.897 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.897 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.897 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.897 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.897 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.898 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.898 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.898 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.898 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.898 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.898 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.898 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.898 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.898 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.898 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.898 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=list 2024-09-16T08:47:28.898 INFO:tasks.workunit.client.0.smithi160.stdout:test list on pid 76699 2024-09-16T08:47:28.906 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76699 2024-09-16T08:47:28.906 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test list on pid 76699' 2024-09-16T08:47:28.906 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76699 2024-09-16T08:47:28.907 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-16T08:47:28.907 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.907 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-09-16T08:47:28.907 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.907 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.907 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.907 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s ec_list 2024-09-16T08:47:28.907 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.907 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.907 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.907 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.907 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' ec_list' 2024-09-16T08:47:28.907 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.907 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.908 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.908 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.908 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.908 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.908 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.908 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.908 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.908 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.908 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.908 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.908 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.908 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.908 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.909 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.909 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.909 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.909 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.909 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.909 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.909 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.909 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.909 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.909 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.909 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.909 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.909 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.909 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.910 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.910 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.910 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.910 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.910 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.910 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.910 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.910 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.910 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.910 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.910 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76512/exe 2024-09-16T08:47:28.910 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.910 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.911 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.911 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.911 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.911 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.911 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.911 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.911 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.911 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.911 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.911 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.911 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.911 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.911 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.911 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.912 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.912 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.912 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.912 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.912 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.912 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.912 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.912 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.912 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.912 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.912 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.912 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.912 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.913 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.913 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.913 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.913 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.913 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.913 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.913 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.913 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.913 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.913 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.913 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.913 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.913 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.914 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.914 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.914 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_cls.log 2024-09-16T08:47:28.914 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ cls: /' 2024-09-16T08:47:28.914 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.914 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.914 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.914 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.914 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.914 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.914 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.914 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.914 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.914 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.915 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.915 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.915 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.915 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.915 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.915 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.915 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.915 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.915 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.915 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.915 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo ec_list 2024-09-16T08:47:28.915 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_cls 2024-09-16T08:47:28.915 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.915 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.916 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.916 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.916 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.916 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.916 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.916 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.916 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.916 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.916 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.916 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.916 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.916 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.916 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.916 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.917 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.917 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.917 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.917 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.917 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.917 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.917 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.917 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.917 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.917 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.917 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.917 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.917 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.918 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.918 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.918 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.918 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.918 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.918 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.918 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.918 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.918 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.918 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.918 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.918 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.918 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.918 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.919 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.919 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.919 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.919 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.919 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.919 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.919 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.919 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.919 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.919 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.919 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.919 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.919 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.919 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.920 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.920 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.920 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.920 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.920 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.920 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.920 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.920 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.920 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.920 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.921 INFO:tasks.workunit.client.0.smithi160.stdout:test ec_list on pid 76764 2024-09-16T08:47:28.925 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=ec_list 2024-09-16T08:47:28.925 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76764 2024-09-16T08:47:28.925 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test ec_list on pid 76764' 2024-09-16T08:47:28.926 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76764 2024-09-16T08:47:28.926 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-16T08:47:28.926 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.926 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.926 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.926 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.926 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.926 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.926 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.926 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.926 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.926 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.926 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s misc 2024-09-16T08:47:28.927 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' misc' 2024-09-16T08:47:28.927 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-09-16T08:47:28.927 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.927 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.927 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.927 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.927 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.927 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.927 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.927 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.927 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.927 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.927 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.928 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.928 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.928 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.928 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.928 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.928 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.928 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.928 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.928 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.928 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.928 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.928 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76590/exe 2024-09-16T08:47:28.928 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.928 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.929 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.929 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.929 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.929 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.929 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.929 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.929 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.929 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.929 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.929 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.929 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.929 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.929 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.930 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.930 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_io 2024-09-16T08:47:28.930 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.930 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.930 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.930 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.930 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.930 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.930 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.930 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.930 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.930 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.930 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.930 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.931 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.931 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.931 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.931 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.931 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.931 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.931 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.931 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_io.log 2024-09-16T08:47:28.931 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.931 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.931 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.931 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.931 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.931 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.932 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.932 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.932 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ io: /' 2024-09-16T08:47:28.932 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.932 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.932 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo misc 2024-09-16T08:47:28.932 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76559/exe 2024-09-16T08:47:28.932 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.932 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.932 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.932 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.932 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.932 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.933 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.933 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.933 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.933 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.933 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.933 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.933 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.933 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.933 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.933 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.933 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.934 INFO:tasks.workunit.client.0.smithi160.stdout:test misc on pid 76806 2024-09-16T08:47:28.936 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=misc 2024-09-16T08:47:28.936 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76806 2024-09-16T08:47:28.936 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test misc on pid 76806' 2024-09-16T08:47:28.936 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76806 2024-09-16T08:47:28.936 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-16T08:47:28.936 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.936 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s pool 2024-09-16T08:47:28.936 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.936 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' pool' 2024-09-16T08:47:28.936 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.936 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.937 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.937 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.937 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.937 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.937 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.937 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.937 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.937 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.937 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.937 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_cmd 2024-09-16T08:47:28.937 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-09-16T08:47:28.937 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ cmd: /' 2024-09-16T08:47:28.938 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_cmd.log 2024-09-16T08:47:28.938 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.938 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.938 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.938 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.938 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.938 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.938 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.938 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.938 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.938 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.938 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.938 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.938 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.939 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.939 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.939 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.939 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.939 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.939 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.939 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.939 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.939 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.939 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.939 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.939 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.939 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.940 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.940 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.940 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.940 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.940 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.940 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.940 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.940 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.940 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.940 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.940 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.940 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.940 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.941 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.941 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.941 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.941 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.941 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.941 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.941 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.941 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.941 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.941 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.941 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.941 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.941 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.941 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.942 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.942 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.942 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.942 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.942 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.942 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.942 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.942 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.942 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.942 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.942 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.943 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.943 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.943 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.943 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.943 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo pool 2024-09-16T08:47:28.943 INFO:tasks.workunit.client.0.smithi160.stdout:test pool on pid 76862 2024-09-16T08:47:28.946 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=pool 2024-09-16T08:47:28.946 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76862 2024-09-16T08:47:28.946 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test pool on pid 76862' 2024-09-16T08:47:28.946 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76862 2024-09-16T08:47:28.946 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-16T08:47:28.946 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.946 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.946 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s read_operations 2024-09-16T08:47:28.946 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' read_operations' 2024-09-16T08:47:28.946 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-09-16T08:47:28.947 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.947 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.947 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.947 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.947 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.947 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.947 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.947 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.947 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.947 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.947 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.947 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.947 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.948 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.948 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.948 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.948 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.948 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.948 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.948 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.948 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.948 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.948 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.948 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.948 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.949 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.949 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.949 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.949 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.949 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.949 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.949 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.949 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.949 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo read_operations 2024-09-16T08:47:28.949 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.949 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.949 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.949 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.949 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.950 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.950 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.950 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.950 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.950 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=read_operations 2024-09-16T08:47:28.950 INFO:tasks.workunit.client.0.smithi160.stdout:test read_operations on pid 76904 2024-09-16T08:47:28.952 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76904 2024-09-16T08:47:28.952 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test read_operations on pid 76904' 2024-09-16T08:47:28.952 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.952 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76904 2024-09-16T08:47:28.952 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-16T08:47:28.952 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.953 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.953 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.953 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.953 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.953 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.953 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.953 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.953 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.953 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.953 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.953 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.953 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-09-16T08:47:28.953 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s snapshots 2024-09-16T08:47:28.954 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' snapshots' 2024-09-16T08:47:28.954 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.954 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.954 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.954 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.954 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.954 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.954 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.954 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.954 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.954 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.954 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.954 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.954 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.955 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.955 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.955 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.955 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.955 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.955 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.955 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.955 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.955 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.955 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.955 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.955 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.955 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.955 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.955 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.956 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.956 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.956 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.956 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.956 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.956 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.956 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.956 INFO:tasks.workunit.client.0.smithi160.stdout:test snapshots on pid 76926 2024-09-16T08:47:28.959 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.959 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo snapshots 2024-09-16T08:47:28.959 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.959 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.959 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.959 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.959 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.959 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.959 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.959 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.959 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.959 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.960 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.960 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.960 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.960 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.960 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.960 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.960 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.960 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.960 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.960 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.960 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.960 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.960 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76643/exe 2024-09-16T08:47:28.960 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=snapshots 2024-09-16T08:47:28.961 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.961 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.961 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.961 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.961 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.961 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.961 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.961 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76926 2024-09-16T08:47:28.961 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test snapshots on pid 76926' 2024-09-16T08:47:28.961 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76926 2024-09-16T08:47:28.961 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.961 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.961 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-16T08:47:28.962 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.962 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.962 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.962 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.962 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.962 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.962 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.962 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.962 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.962 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.962 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.962 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.962 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-09-16T08:47:28.963 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.963 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.963 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.963 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.963 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76699/exe 2024-09-16T08:47:28.963 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.963 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.963 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.963 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.963 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.963 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.963 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.963 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.963 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.964 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.964 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.964 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.964 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.964 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.964 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.964 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.964 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.964 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.964 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.964 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.964 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.964 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.964 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.965 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.965 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.965 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.965 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.965 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.965 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.965 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.965 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.965 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.965 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.965 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.965 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.965 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.965 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.966 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.966 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.966 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.966 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.966 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s watch_notify 2024-09-16T08:47:28.966 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.966 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' watch_notify' 2024-09-16T08:47:28.966 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.966 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.966 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.966 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.966 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.966 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.966 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.967 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.967 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.967 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.967 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.967 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo watch_notify 2024-09-16T08:47:28.967 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_ec_io 2024-09-16T08:47:28.967 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.967 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.967 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.967 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.967 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.967 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.967 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.967 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.968 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.968 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.968 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.968 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.968 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.968 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.968 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.968 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.968 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.968 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.968 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.968 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.968 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.968 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.969 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.969 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.969 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.969 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.969 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.969 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.969 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_ec_io.log 2024-09-16T08:47:28.969 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.969 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.969 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.969 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.969 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.969 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.969 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.970 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.970 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.970 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.970 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.970 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.970 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.970 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.970 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.970 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.970 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_list.log 2024-09-16T08:47:28.970 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.970 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.970 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.970 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.971 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.971 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.971 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.971 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.971 INFO:tasks.workunit.client.0.smithi160.stdout:test watch_notify on pid 76977 2024-09-16T08:47:28.977 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=watch_notify 2024-09-16T08:47:28.977 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76977 2024-09-16T08:47:28.977 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test watch_notify on pid 76977' 2024-09-16T08:47:28.977 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=76977 2024-09-16T08:47:28.978 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-16T08:47:28.978 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:28.978 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.978 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ list: /' 2024-09-16T08:47:28.978 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_list 2024-09-16T08:47:28.978 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.978 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.978 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.978 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.978 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.978 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.978 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.978 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.979 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.979 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ ec_io: /' 2024-09-16T08:47:28.979 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-09-16T08:47:28.979 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.979 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.979 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.979 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.979 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.979 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.979 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.979 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.979 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.979 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.979 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.980 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.980 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.980 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.980 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.980 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.980 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.980 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.980 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.980 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.980 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.980 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.980 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.980 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.981 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.981 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.981 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.981 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.981 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.981 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.981 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.981 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.981 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.981 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.981 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.981 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:28.981 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:28.981 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:28.982 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:28.982 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.982 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.982 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:28.982 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:28.982 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:28.982 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:28.982 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.982 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:28.982 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.982 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:28.982 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.982 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.982 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.983 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:28.983 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.983 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.983 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.983 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:28.983 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.983 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:28.983 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.983 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:28.983 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.983 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:28.983 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.983 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:28.984 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.984 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.984 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.984 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.984 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.984 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.984 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.984 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.984 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.984 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.984 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.984 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.984 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.985 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.985 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.985 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.985 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.985 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.985 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.985 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.985 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:28.985 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:28.985 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.985 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:28.985 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.985 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.986 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.986 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.986 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.986 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.986 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.986 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.986 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.986 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.986 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.986 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.986 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.986 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.986 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.986 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.987 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:28.987 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.987 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.987 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.987 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:28.987 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.987 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.987 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:28.987 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.987 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:28.987 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.987 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:28.987 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.987 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:28.988 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.988 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:28.988 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:28.988 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:28.988 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.988 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.988 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.988 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.988 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.988 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.988 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.988 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.988 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.988 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.989 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.989 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.989 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.989 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.989 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.989 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.989 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.989 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.989 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.989 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.989 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.989 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.990 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.990 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.990 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.990 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.990 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s write_operations 2024-09-16T08:47:28.990 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' write_operations' 2024-09-16T08:47:28.990 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.990 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.990 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.990 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.990 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.990 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.990 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.990 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.991 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.991 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.991 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.991 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.991 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.991 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:28.991 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:28.991 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.991 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.991 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:28.991 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.991 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.991 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.991 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:28.992 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:28.992 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:28.992 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.992 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:28.992 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.992 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.992 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:28.992 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.992 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.992 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.992 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.992 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:28.992 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.992 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.993 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76806/exe 2024-09-16T08:47:28.993 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo write_operations 2024-09-16T08:47:28.993 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:28.993 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:28.993 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:28.993 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:28.993 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:28.993 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.993 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:28.993 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:28.993 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.993 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:28.993 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:28.993 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:28.994 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.994 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:28.994 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.994 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:28.994 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:28.994 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:28.994 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.994 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:28.994 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.994 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:28.994 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:28.994 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-16T08:47:28.994 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76904/exe 2024-09-16T08:47:28.994 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:28.995 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:28.995 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:28.995 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:28.995 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:28.995 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:28.995 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:28.995 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:28.995 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:28.995 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.995 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:28.995 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.995 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:28.995 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:28.995 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:28.996 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:28.996 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:28.996 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.996 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:28.996 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:28.996 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:28.996 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.996 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:28.996 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:28.996 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:28.996 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:28.996 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:28.996 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:28.997 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:28.997 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:28.997 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ misc: /' 2024-09-16T08:47:28.997 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76764/exe 2024-09-16T08:47:28.997 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_misc.log 2024-09-16T08:47:28.997 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:28.997 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_misc 2024-09-16T08:47:28.997 INFO:tasks.workunit.client.0.smithi160.stdout:test write_operations on pid 77048 2024-09-16T08:47:29.005 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:29.005 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:29.005 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:29.005 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=write_operations 2024-09-16T08:47:29.005 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:29.005 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:29.005 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:29.006 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:29.006 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:29.006 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:29.006 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:29.006 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:29.006 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:29.006 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=77048 2024-09-16T08:47:29.006 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.006 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test write_operations on pid 77048' 2024-09-16T08:47:29.006 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:29.006 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.006 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=77048 2024-09-16T08:47:29.006 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:29.006 INFO:tasks.workunit.client.0.smithi160.stderr:+ ret=0 2024-09-16T08:47:29.007 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:29.007 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.007 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:29.007 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-16T08:47:29.007 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.007 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:29.007 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:29.007 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T08:47:29.007 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75843 2024-09-16T08:47:29.007 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-09-16T08:47:29.007 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 75843 2024-09-16T08:47:29.007 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:29.007 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:29.008 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:29.008 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:29.008 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:29.008 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:29.008 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:29.008 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:29.008 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:29.008 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:29.008 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -f /etc/bashrc ']' 2024-09-16T08:47:29.008 INFO:tasks.workunit.client.0.smithi160.stderr:+ . /etc/bashrc 2024-09-16T08:47:29.008 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ read_operations: /' 2024-09-16T08:47:29.008 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -z '' ']' 2024-09-16T08:47:29.008 INFO:tasks.workunit.client.0.smithi160.stderr:++ BASHRCSOURCED=Y 2024-09-16T08:47:29.008 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.009 INFO:tasks.workunit.client.0.smithi160.stderr:++ shopt -q login_shell 2024-09-16T08:47:29.009 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_read_operations.log 2024-09-16T08:47:29.009 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76862/exe 2024-09-16T08:47:29.009 INFO:tasks.workunit.client.0.smithi160.stderr:+++ umask 2024-09-16T08:47:29.009 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:29.009 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:29.009 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:29.009 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:29.009 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:29.009 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:29.009 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:29.009 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' 0022 -eq 0 ']' 2024-09-16T08:47:29.009 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:29.009 INFO:tasks.workunit.client.0.smithi160.stderr:++ SHELL=/bin/bash 2024-09-16T08:47:29.010 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.010 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:29.010 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-16T08:47:29.010 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.010 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-16T08:47:29.010 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:29.010 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:29.010 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-16T08:47:29.010 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.010 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-16T08:47:29.010 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.010 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-16T08:47:29.010 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:29.010 INFO:tasks.workunit.client.0.smithi160.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-16T08:47:29.011 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.011 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-16T08:47:29.011 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.011 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-16T08:47:29.011 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:29.011 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:29.011 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:29.011 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:29.011 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:29.011 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:29.011 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:29.011 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:29.011 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:29.012 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:29.012 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:29.012 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.012 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:29.012 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.012 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:29.012 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:29.012 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:29.012 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:29.012 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:29.012 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:29.012 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_ec_list.log 2024-09-16T08:47:29.012 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ ec_list: /' 2024-09-16T08:47:29.013 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_ec_list 2024-09-16T08:47:29.013 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_read_operations 2024-09-16T08:47:29.013 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:29.013 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:29.013 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:29.013 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.013 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:29.013 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.013 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:29.013 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:29.013 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.013 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:29.013 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.013 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:29.014 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.014 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:29.014 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.014 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:29.014 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:29.014 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:29.014 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_pool 2024-09-16T08:47:29.014 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:29.014 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:29.014 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:29.014 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.014 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:29.014 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.015 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:29.015 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:29.015 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.015 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:29.015 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.015 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:29.015 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.015 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:29.015 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.015 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:29.015 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:29.015 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:29.015 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:29.015 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:29.016 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:29.016 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:29.016 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:29.016 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_pool.log 2024-09-16T08:47:29.016 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:29.016 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:29.016 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ pool: /' 2024-09-16T08:47:29.016 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:29.016 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:29.016 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:29.016 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:29.016 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'grep=grep --color=auto' 2024-09-16T08:47:29.016 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-16T08:47:29.016 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-16T08:47:29.017 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.017 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-16T08:47:29.017 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.017 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorls.sh 2024-09-16T08:47:29.017 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' '!' -t 0 ']' 2024-09-16T08:47:29.017 INFO:tasks.workunit.client.0.smithi160.stderr:+++ return 2024-09-16T08:47:29.017 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.017 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-16T08:47:29.017 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.017 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-16T08:47:29.017 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export S_COLORS=auto 2024-09-16T08:47:29.017 INFO:tasks.workunit.client.0.smithi160.stderr:+++ S_COLORS=auto 2024-09-16T08:47:29.017 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.018 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-16T08:47:29.018 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.018 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-16T08:47:29.018 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:29.018 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:29.018 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:29.020 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:29.020 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:29.020 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:29.020 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:29.020 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:29.020 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:29.020 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:29.020 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:29.020 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:29.020 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.020 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:29.021 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.021 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:29.021 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:29.021 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.021 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:29.021 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.021 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:29.021 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:29.021 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:29.021 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:29.021 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:29.021 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:29.021 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:29.022 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:29.022 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:29.022 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:29.022 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:29.022 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:29.022 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:29.022 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:29.022 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.022 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:29.022 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.022 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:29.022 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-16T08:47:29.022 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:29.022 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.023 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:29.023 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-16T08:47:29.023 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.023 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:29.023 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-16T08:47:29.023 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.023 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:29.023 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-16T08:47:29.023 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.023 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-16T08:47:29.023 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-16T08:47:29.023 INFO:tasks.workunit.client.0.smithi160.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-16T08:47:29.023 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76926/exe 2024-09-16T08:47:29.023 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/76977/exe 2024-09-16T08:47:29.024 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:29.024 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:29.024 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:29.024 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:29.024 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:29.024 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:29.024 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:29.024 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:29.024 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:29.024 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:29.024 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:29.024 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:29.024 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:29.024 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:29.025 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:29.025 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:29.025 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:29.025 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:29.025 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:29.025 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:29.025 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:29.025 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:29.025 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ snapshots: /' 2024-09-16T08:47:29.025 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-16T08:47:29.025 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-16T08:47:29.025 INFO:tasks.workunit.client.0.smithi160.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-16T08:47:29.025 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.026 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-16T08:47:29.026 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.026 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-16T08:47:29.026 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-16T08:47:29.026 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.026 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-16T08:47:29.026 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.026 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/gawk.sh 2024-09-16T08:47:29.026 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.026 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-16T08:47:29.026 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.026 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/lang.sh 2024-09-16T08:47:29.026 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup 2024-09-16T08:47:29.026 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_snapshots 2024-09-16T08:47:29.027 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-09-16T08:47:29.027 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/locale 2024-09-16T08:47:29.027 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_snapshots.log 2024-09-16T08:47:29.027 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:29.027 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-16T08:47:29.027 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:29.027 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-16T08:47:29.027 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-16T08:47:29.027 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_watch_notify 2024-09-16T08:47:29.027 INFO:tasks.workunit.client.0.smithi160.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-16T08:47:29.027 INFO:tasks.workunit.client.0.smithi160.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-16T08:47:29.027 INFO:tasks.workunit.client.0.smithi160.stderr:++++ export LANG=en_US.UTF-8 2024-09-16T08:47:29.027 INFO:tasks.workunit.client.0.smithi160.stderr:++++ LANG=en_US.UTF-8 2024-09-16T08:47:29.027 INFO:tasks.workunit.client.0.smithi160.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-16T08:47:29.028 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-16T08:47:29.028 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:29.028 INFO:tasks.workunit.client.0.smithi160.stderr:+++ LANG=en_US.UTF-8 2024-09-16T08:47:29.028 INFO:tasks.workunit.client.0.smithi160.stderr:+++ unset LANG_backup config 2024-09-16T08:47:29.028 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n '' ']' 2024-09-16T08:47:29.028 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-16T08:47:29.028 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' dumb = linux ']' 2024-09-16T08:47:29.028 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.028 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-16T08:47:29.028 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.028 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/less.sh 2024-09-16T08:47:29.028 INFO:tasks.workunit.client.0.smithi160.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-16T08:47:29.028 INFO:tasks.workunit.client.0.smithi160.stderr:++ for i in /etc/profile.d/*.sh 2024-09-16T08:47:29.028 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-16T08:47:29.029 INFO:tasks.workunit.client.0.smithi160.stderr:++ '[' '' ']' 2024-09-16T08:47:29.029 INFO:tasks.workunit.client.0.smithi160.stderr:++ . /etc/profile.d/which2.sh 2024-09-16T08:47:29.029 INFO:tasks.workunit.client.0.smithi160.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-16T08:47:29.029 INFO:tasks.workunit.client.0.smithi160.stderr:+++++ readlink /proc/77048/exe 2024-09-16T08:47:29.029 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ watch_notify: /' 2024-09-16T08:47:29.029 INFO:tasks.workunit.client.0.smithi160.stderr:++++ basename /usr/bin/bash 2024-09-16T08:47:29.029 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_declare='declare -f' 2024-09-16T08:47:29.029 INFO:tasks.workunit.client.0.smithi160.stderr:+++ which_opt=-f 2024-09-16T08:47:29.029 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export which_declare 2024-09-16T08:47:29.029 INFO:tasks.workunit.client.0.smithi160.stderr:+++ export -f which 2024-09-16T08:47:29.029 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset i 2024-09-16T08:47:29.029 INFO:tasks.workunit.client.0.smithi160.stderr:++ unset -f pathmunge 2024-09-16T08:47:29.029 INFO:tasks.workunit.client.0.smithi160.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-16T08:47:29.029 INFO:tasks.workunit.client.0.smithi160.stderr:+ export PATH 2024-09-16T08:47:29.030 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-16T08:47:29.030 INFO:tasks.workunit.client.0.smithi160.stderr:+ unset rc 2024-09-16T08:47:29.030 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_write_operations 2024-09-16T08:47:29.030 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ write_operations: /' 2024-09-16T08:47:29.030 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_write_operations.log 2024-09-16T08:47:29.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:29.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[40028]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:29.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:29.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[40028]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:29.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[40028]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1018083209' entity='client.admin' cmd={"prefix":"quorum_status"} : dispatch 2024-09-16T08:47:29.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2315144845' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi160-76956-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[40028]: from='client.24898 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi160-76956-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/985108347' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi160-77061-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[40028]: from='client.24913 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi160-77061-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:29.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[30137]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:29.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:29.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[30137]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:29.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[30137]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1018083209' entity='client.admin' cmd={"prefix":"quorum_status"} : dispatch 2024-09-16T08:47:29.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2315144845' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi160-76956-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[30137]: from='client.24898 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi160-76956-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/985108347' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi160-77061-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:29 smithi160 ceph-mon[30137]: from='client.24913 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi160-77061-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:29.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:29 smithi169 ceph-mon[33465]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:29.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:29.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:29 smithi169 ceph-mon[33465]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:29.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:29 smithi169 ceph-mon[33465]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1018083209' entity='client.admin' cmd={"prefix":"quorum_status"} : dispatch 2024-09-16T08:47:29.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2315144845' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi160-76956-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:29 smithi169 ceph-mon[33465]: from='client.24898 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi160-76956-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/985108347' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi160-77061-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:29.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:29 smithi169 ceph-mon[33465]: from='client.24913 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi160-77061-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:30.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: pgmap v48: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:47:30.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.34769 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:30.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24898 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi160-76956-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:30.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi160-77061-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:30.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T08:47:30.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2315144845' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi160-76956-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:30.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/985108347' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi160-77061-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:30.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24898 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi160-76956-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:30.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/130253646' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi160-76059-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24913 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi160-77061-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:30.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2564608026' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi160-75783-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3516229818' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi160-76301-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2417530941' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi160-75849-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/38231908' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi160-75757-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/991638596' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi160-75791-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi160-75864-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2608321' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76320-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24748 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi160-76059-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi160-76317-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:30.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24700 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi160-75783-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi160-76317-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:30.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1702248129' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24772 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi160-76301-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24724 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi160-75849-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1121940498' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/32879386' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi160-76142-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/140304623' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi160-76516-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1907724577' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76376-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1721895572' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi160-75827-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1824387761' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi160-76133-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/851199487' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi160-76079-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1099269235' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi160-75741-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24694 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi160-75757-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.34697 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi160-75827-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24709 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi160-75791-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.34739 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi160-76133-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24730 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi160-75864-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.34751 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi160-76079-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24871 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76320-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.34679 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi160-75741-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/6350294' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi160-76254-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.34757 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi160-76254-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24745 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24751 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24760 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi160-76142-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24820 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi160-76516-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[30137]: from='client.24868 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76376-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: pgmap v48: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:47:30.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.34769 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:30.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24898 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi160-76956-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:30.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi160-77061-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:30.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T08:47:30.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2315144845' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi160-76956-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:30.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/985108347' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi160-77061-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:30.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24898 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi160-76956-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:30.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/130253646' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi160-76059-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24913 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi160-77061-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:30.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2564608026' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi160-75783-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3516229818' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi160-76301-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2417530941' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi160-75849-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/38231908' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi160-75757-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/991638596' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi160-75791-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi160-75864-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2608321' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76320-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24748 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi160-76059-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi160-76317-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:30.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24700 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi160-75783-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi160-76317-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:30.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1702248129' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24772 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi160-76301-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24724 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi160-75849-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1121940498' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/32879386' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi160-76142-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/140304623' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi160-76516-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1907724577' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76376-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1721895572' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi160-75827-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1824387761' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi160-76133-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/851199487' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi160-76079-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1099269235' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi160-75741-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24694 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi160-75757-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.34697 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi160-75827-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24709 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi160-75791-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.34739 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi160-76133-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24730 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi160-75864-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.34751 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi160-76079-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24871 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76320-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.34679 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi160-75741-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/6350294' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi160-76254-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.34757 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi160-76254-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24745 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24751 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24760 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi160-76142-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24820 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi160-76516-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:30 smithi160 ceph-mon[40028]: from='client.24868 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76376-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.548 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-09-16T08:47:30.549 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [----------] Global test environment set-up. 2024-09-16T08:47:30.549 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [----------] 12 tests from AsioRados 2024-09-16T08:47:30.549 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-09-16T08:47:30.549 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (1 ms) 2024-09-16T08:47:30.549 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-09-16T08:47:30.549 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (1 ms) 2024-09-16T08:47:30.549 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-09-16T08:47:30.549 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (0 ms) 2024-09-16T08:47:30.549 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-09-16T08:47:30.549 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (2 ms) 2024-09-16T08:47:30.549 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-09-16T08:47:30.549 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (5 ms) 2024-09-16T08:47:30.549 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-09-16T08:47:30.550 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (5 ms) 2024-09-16T08:47:30.550 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-09-16T08:47:30.550 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-09-16T08:47:30.550 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-09-16T08:47:30.550 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (1 ms) 2024-09-16T08:47:30.550 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-09-16T08:47:30.550 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (1 ms) 2024-09-16T08:47:30.550 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-09-16T08:47:30.550 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (4 ms) 2024-09-16T08:47:30.550 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-09-16T08:47:30.550 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (1 ms) 2024-09-16T08:47:30.550 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-09-16T08:47:30.550 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (3 ms) 2024-09-16T08:47:30.550 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [----------] 12 tests from AsioRados (25 ms total) 2024-09-16T08:47:30.551 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-16T08:47:30.551 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [----------] Global test environment tear-down 2024-09-16T08:47:30.551 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (1972 ms total) 2024-09-16T08:47:30.551 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ PASSED ] 12 tests. 2024-09-16T08:47:30.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: pgmap v48: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:47:30.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.34769 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:30.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24898 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi160-76956-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi160-77061-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T08:47:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2315144845' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi160-76956-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/985108347' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi160-77061-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24898 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi160-76956-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/130253646' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi160-76059-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24913 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi160-77061-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2564608026' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi160-75783-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3516229818' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi160-76301-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2417530941' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi160-75849-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/38231908' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi160-75757-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/991638596' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi160-75791-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi160-75864-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2608321' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76320-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24748 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi160-76059-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi160-76317-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:30.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24700 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi160-75783-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi160-76317-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:30.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1702248129' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24772 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi160-76301-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24724 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi160-75849-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1121940498' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/32879386' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi160-76142-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/140304623' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi160-76516-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1907724577' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76376-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1721895572' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi160-75827-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1824387761' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi160-76133-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/851199487' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi160-76079-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1099269235' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi160-75741-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24694 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi160-75757-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.34697 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi160-75827-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24709 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi160-75791-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.34739 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi160-76133-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24730 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi160-75864-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.34751 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi160-76079-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24871 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76320-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.34679 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi160-75741-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/6350294' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi160-76254-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.34757 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi160-76254-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24745 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24751 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24760 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi160-76142-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24820 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi160-76516-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:30 smithi169 ceph-mon[33465]: from='client.24868 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76376-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:30.881 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-09-16T08:47:30.881 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [----------] Global test environment set-up. 2024-09-16T08:47:30.882 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [----------] 7 tests from LibRadosList 2024-09-16T08:47:30.882 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-09-16T08:47:30.882 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosList.ListObjects (450 ms) 2024-09-16T08:47:30.882 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-09-16T08:47:30.882 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (37 ms) 2024-09-16T08:47:30.882 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-09-16T08:47:30.882 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: myset foo1,foo2,foo3 2024-09-16T08:47:30.882 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo1 2024-09-16T08:47:30.882 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo2 2024-09-16T08:47:30.883 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo3 2024-09-16T08:47:30.883 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: myset foo1,foo4,foo5 2024-09-16T08:47:30.883 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo4 2024-09-16T08:47:30.883 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo5 2024-09-16T08:47:30.883 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo1 2024-09-16T08:47:30.883 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: myset foo6,foo7 2024-09-16T08:47:30.883 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo7 2024-09-16T08:47:30.883 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo6 2024-09-16T08:47:30.883 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-16T08:47:30.883 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns1:foo4 2024-09-16T08:47:30.883 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns1:foo5 2024-09-16T08:47:30.883 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns2:foo7 2024-09-16T08:47:30.883 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns2:foo6 2024-09-16T08:47:30.883 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns1:foo1 2024-09-16T08:47:30.883 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: :foo1 2024-09-16T08:47:30.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: :foo2 2024-09-16T08:47:30.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: :foo3 2024-09-16T08:47:30.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (94 ms) 2024-09-16T08:47:30.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-09-16T08:47:30.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 1 0 2024-09-16T08:47:30.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 10 0 2024-09-16T08:47:30.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 13 0 2024-09-16T08:47:30.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 7 0 2024-09-16T08:47:30.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 14 0 2024-09-16T08:47:30.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 0 0 2024-09-16T08:47:30.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 15 0 2024-09-16T08:47:30.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 11 0 2024-09-16T08:47:30.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 5 0 2024-09-16T08:47:30.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 8 0 2024-09-16T08:47:30.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 6 0 2024-09-16T08:47:30.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 3 0 2024-09-16T08:47:30.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 4 0 2024-09-16T08:47:30.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 12 0 2024-09-16T08:47:30.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 9 0 2024-09-16T08:47:30.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 2 0 2024-09-16T08:47:30.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-16T08:47:30.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (71 ms) 2024-09-16T08:47:30.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-09-16T08:47:30.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: x cursor=MIN 2024-09-16T08:47:30.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=1 cursor=14:02547ec2:::1:head 2024-09-16T08:47:30.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=10 cursor=14:52ea6a34:::10:head 2024-09-16T08:47:30.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=13 cursor=14:566253c9:::13:head 2024-09-16T08:47:30.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=7 cursor=14:5c6b0b28:::7:head 2024-09-16T08:47:30.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=14 cursor=14:62a1935d:::14:head 2024-09-16T08:47:30.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=0 cursor=14:6cac518f:::0:head 2024-09-16T08:47:30.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=15 cursor=14:863748b0:::15:head 2024-09-16T08:47:30.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=11 cursor=14:89d3ae78:::11:head 2024-09-16T08:47:30.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=5 cursor=14:b29083e3:::5:head 2024-09-16T08:47:30.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=8 cursor=14:bd63b0f1:::8:head 2024-09-16T08:47:30.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=6 cursor=14:c4fdafeb:::6:head 2024-09-16T08:47:30.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=3 cursor=14:cfc208b3:::3:head 2024-09-16T08:47:30.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=4 cursor=14:d83876eb:::4:head 2024-09-16T08:47:30.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=12 cursor=14:de5d7c5f:::12:head 2024-09-16T08:47:30.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=9 cursor=14:e960b815:::9:head 2024-09-16T08:47:30.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=2 cursor=14:f905c69b:::2:head 2024-09-16T08:47:30.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: FIRST> seek to MIN oid=1 2024-09-16T08:47:30.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=1 cursor=14:02547ec2:::1:head 2024-09-16T08:47:30.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:02547ec2:::1:head 2024-09-16T08:47:30.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:02547ec2:::1:head -> 1 2024-09-16T08:47:30.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=10 cursor=14:52ea6a34:::10:head 2024-09-16T08:47:30.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:52ea6a34:::10:head 2024-09-16T08:47:30.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:52ea6a34:::10:head -> 10 2024-09-16T08:47:30.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=13 cursor=14:566253c9:::13:head 2024-09-16T08:47:30.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:566253c9:::13:head 2024-09-16T08:47:30.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:566253c9:::13:head -> 13 2024-09-16T08:47:30.888 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=7 cursor=14:5c6b0b28:::7:head 2024-09-16T08:47:30.888 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:5c6b0b28:::7:head 2024-09-16T08:47:30.888 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:5c6b0b28:::7:head -> 7 2024-09-16T08:47:30.954 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=14 cursor=14:62a1935d:::14:head 2024-09-16T08:47:30.954 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:62a1935d:::14:head 2024-09-16T08:47:30.954 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:62a1935d:::14:head -> 14 2024-09-16T08:47:30.954 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=0 cursor=14:6cac518f:::0:head 2024-09-16T08:47:30.954 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:6cac518f:::0:head 2024-09-16T08:47:30.954 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:6cac518f:::0:head -> 0 2024-09-16T08:47:30.954 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=15 cursor=14:863748b0:::15:head 2024-09-16T08:47:30.954 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:863748b0:::15:head 2024-09-16T08:47:30.954 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:863748b0:::15:head -> 15 2024-09-16T08:47:30.954 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=11 cursor=14:89d3ae78:::11:head 2024-09-16T08:47:30.954 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:89d3ae78:::11:head 2024-09-16T08:47:30.955 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:89d3ae78:::11:head -> 11 2024-09-16T08:47:30.955 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=5 cursor=14:b29083e3:::5:head 2024-09-16T08:47:30.955 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:b29083e3:::5:head 2024-09-16T08:47:30.955 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:b29083e3:::5:head -> 5 2024-09-16T08:47:30.955 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=8 cursor=14:bd63b0f1:::8:head 2024-09-16T08:47:30.955 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:bd63b0f1:::8:head 2024-09-16T08:47:30.955 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:bd63b0f1:::8:head -> 8 2024-09-16T08:47:30.955 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=6 cursor=14:c4fdafeb:::6:head 2024-09-16T08:47:30.955 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:c4fdafeb:::6:head 2024-09-16T08:47:30.955 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:c4fdafeb:::6:head -> 6 2024-09-16T08:47:30.955 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=3 cursor=14:cfc208b3:::3:head 2024-09-16T08:47:30.955 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:cfc208b3:::3:head 2024-09-16T08:47:30.955 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:cfc208b3:::3:head -> 3 2024-09-16T08:47:30.955 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=4 cursor=14:d83876eb:::4:head 2024-09-16T08:47:30.956 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:d83876eb:::4:head 2024-09-16T08:47:30.956 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:d83876eb:::4:head -> 4 2024-09-16T08:47:30.956 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=12 cursor=14:de5d7c5f:::12:head 2024-09-16T08:47:30.956 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:de5d7c5f:::12:head 2024-09-16T08:47:30.956 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:de5d7c5f:::12:head -> 12 2024-09-16T08:47:30.956 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=9 cursor=14:e960b815:::9:head 2024-09-16T08:47:30.956 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:e960b815:::9:head 2024-09-16T08:47:30.956 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:e960b815:::9:head -> 9 2024-09-16T08:47:30.956 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=2 cursor=14:f905c69b:::2:head 2024-09-16T08:47:30.956 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:f905c69b:::2:head 2024-09-16T08:47:30.956 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:f905c69b:::2:head -> 2 2024-09-16T08:47:30.956 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:52ea6a34:::10:head 2024-09-16T08:47:30.956 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=14:52ea6a34:::10:head expected=14:52ea6a34:::10:head 2024-09-16T08:47:30.957 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:52ea6a34:::10:head -> 10 2024-09-16T08:47:30.957 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=10 expected=10 2024-09-16T08:47:30.957 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:d83876eb:::4:head 2024-09-16T08:47:30.957 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=14:d83876eb:::4:head expected=14:d83876eb:::4:head 2024-09-16T08:47:30.957 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:d83876eb:::4:head -> 4 2024-09-16T08:47:30.957 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=4 expected=4 2024-09-16T08:47:30.957 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:cfc208b3:::3:head 2024-09-16T08:47:30.957 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=14:cfc208b3:::3:head expected=14:cfc208b3:::3:head 2024-09-16T08:47:30.957 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:cfc208b3:::3:head -> 3 2024-09-16T08:47:30.957 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=3 expected=3 2024-09-16T08:47:30.958 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:c4fdafeb:::6:head 2024-09-16T08:47:30.958 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=14:c4fdafeb:::6:head expected=14:c4fdafeb:::6:head 2024-09-16T08:47:30.958 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:c4fdafeb:::6:head -> 6 2024-09-16T08:47:30.958 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=6 expected=6 2024-09-16T08:47:30.958 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:bd63b0f1:::8:head 2024-09-16T08:47:30.958 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=14:bd63b0f1:::8:head expected=14:bd63b0f1:::8:head 2024-09-16T08:47:30.958 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:bd63b0f1:::8:head -> 8 2024-09-16T08:47:30.958 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=8 expected=8 2024-09-16T08:47:30.958 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:b29083e3:::5:head 2024-09-16T08:47:30.958 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=14:b29083e3:::5:head expected=14:b29083e3:::5:head 2024-09-16T08:47:30.958 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:b29083e3:::5:head -> 5 2024-09-16T08:47:30.958 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=5 expected=5 2024-09-16T08:47:30.958 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:02547ec2:::1:head 2024-09-16T08:47:30.958 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=14:02547ec2:::1:head expected=14:02547ec2:::1:head 2024-09-16T08:47:30.959 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:02547ec2:::1:head -> 1 2024-09-16T08:47:30.959 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=1 expected=1 2024-09-16T08:47:30.959 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:5c6b0b28:::7:head 2024-09-16T08:47:30.959 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=14:5c6b0b28:::7:head expected=14:5c6b0b28:::7:head 2024-09-16T08:47:30.959 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:5c6b0b28:::7:head -> 7 2024-09-16T08:47:31.095 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=7 expecte list_parallel: process_1_[76606]: starting. 2024-09-16T08:47:31.095 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_1_[76606]: creating pool ceph_test_rados_list_parallel.smithi160-76567 2024-09-16T08:47:31.095 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_1_[76606]: created object 0... 2024-09-16T08:47:31.095 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_1_[76606]: created object 25... 2024-09-16T08:47:31.095 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_1_[76606]: created object 49... 2024-09-16T08:47:31.095 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_1_[76606]: finishing. 2024-09-16T08:47:31.096 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_1_[76606]: shutting down. 2024-09-16T08:47:31.096 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_2_[76609]: starting. 2024-09-16T08:47:31.096 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_2_[76609]: listing objects. 2024-09-16T08:47:31.096 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_2_[76609]: listed object 0... 2024-09-16T08:47:31.096 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_2_[76609]: listed object 25... 2024-09-16T08:47:31.096 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_2_[76609]: saw 50 objects 2024-09-16T08:47:31.096 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_2_[76609]: shutting down. 2024-09-16T08:47:31.096 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.096 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_3_[77354]: starting. 2024-09-16T08:47:31.096 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_3_[77354]: creating pool ceph_test_rados_list_parallel.smithi160-76567 2024-09-16T08:47:31.097 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_3_[77354]: created object 0... 2024-09-16T08:47:31.097 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_3_[77354]: created object 25... 2024-09-16T08:47:31.097 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_3_[77354]: created object 49... 2024-09-16T08:47:31.097 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_3_[77354]: finishing. 2024-09-16T08:47:31.097 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_3_[77354]: shutting down. 2024-09-16T08:47:31.097 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.097 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_4_[77355]: starting. 2024-09-16T08:47:31.097 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_4_[77355]: listing objects. 2024-09-16T08:47:31.097 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_4_[77355]: listed object 0... 2024-09-16T08:47:31.097 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_4_[77355]: listed object 25... 2024-09-16T08:47:31.097 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_4_[77355]: saw 43 objects 2024-09-16T08:47:31.097 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_4_[77355]: shutting down. 2024-09-16T08:47:31.097 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.097 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_5_[77356]: starting. 2024-09-16T08:47:31.098 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_5_[77356]: removed 25 objects... 2024-09-16T08:47:31.098 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_5_[77356]: removed half of the objects 2024-09-16T08:47:31.098 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_5_[77356]: removed 50 objects... 2024-09-16T08:47:31.098 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_5_[77356]: removed 50 objects 2024-09-16T08:47:31.098 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_5_[77356]: shutting down. 2024-09-16T08:47:31.098 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.098 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.098 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_6_[77403]: starting. 2024-09-16T08:47:31.098 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_6_[77403]: creating pool ceph_test_rados_list_parallel.smithi160-76567 2024-09-16T08:47:31.098 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_6_[77403]: created object 0... 2024-09-16T08:47:31.098 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_6_[77403]: created object 25... 2024-09-16T08:47:31.098 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_6_[77403]: created object 49... 2024-09-16T08:47:31.098 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_6_[77403]: finishing. 2024-09-16T08:47:31.098 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_6_[77403]: shutting down. 2024-09-16T08:47:31.099 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.099 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.099 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_7_[77404]: starting. 2024-09-16T08:47:31.099 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_7_[77404]: listing objects. 2024-09-16T08:47:31.099 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_7_[77404]: listed object 0... 2024-09-16T08:47:31.099 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_7_[77404]: listed object 25... 2024-09-16T08:47:31.099 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_7_[77404]: listed object 50... 2024-09-16T08:47:31.099 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_7_[77404]: saw 56 objects 2024-09-16T08:47:31.099 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_7_[77404]: shutting down. 2024-09-16T08:47:31.099 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.099 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.099 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_8_[77405]: starting. 2024-09-16T08:47:31.099 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_8_[77405]: added 25 objects... 2024-09-16T08:47:31.099 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_8_[77405]: added half of the objects 2024-09-16T08:47:31.100 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_8_[77405]: added 50 objects... 2024-09-16T08:47:31.100 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_8_[77405]: added 50 objects 2024-09-16T08:47:31.100 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_8_[77405]: shutting down. 2024-09-16T08:47:31.100 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.100 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.170 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ********************** cls: Running main() from gmock_main.cc 2024-09-16T08:47:31.170 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [==========] Running 1 test from 1 test suite. 2024-09-16T08:47:31.170 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [----------] Global test environment set-up. 2024-09-16T08:47:31.171 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [----------] 1 test from NeoRadosCls 2024-09-16T08:47:31.171 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-09-16T08:47:31.171 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [ OK ] NeoRadosCls.DNE (2322 ms) 2024-09-16T08:47:31.171 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [----------] 1 test from NeoRadosCls (2322 ms total) 2024-09-16T08:47:31.171 INFO:tasks.workunit.client.0.smithi160.stdout: cls: 2024-09-16T08:47:31.171 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [----------] Global test environment tear-down 2024-09-16T08:47:31.171 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [==========] 1 test from 1 test suite ran. (2322 ms total) 2024-09-16T08:47:31.171 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [ PASSED ] 1 test. 2024-09-16T08:47:31.176 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-09-16T08:47:31.176 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-09-16T08:47:31.176 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-09-16T08:47:31.176 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-09-16T08:47:31.176 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-09-16T08:47:31.176 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (46 ms) 2024-09-16T08:47:31.176 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-09-16T08:47:31.177 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (74 ms) 2024-09-16T08:47:31.177 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-09-16T08:47:31.177 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (2326 ms) 2024-09-16T08:47:31.177 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (2446 ms total) 2024-09-16T08:47:31.177 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: 2024-09-16T08:47:31.177 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-09-16T08:47:31.177 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (2446 ms total) 2024-09-16T08:47:31.177 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-09-16T08:47:31.182 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: Running main() from gmock_main.cc 2024-09-16T08:47:31.183 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-09-16T08:47:31.183 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [----------] Global test environment set-up. 2024-09-16T08:47:31.183 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-09-16T08:47:31.183 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-09-16T08:47:31.183 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (2345 ms) 2024-09-16T08:47:31.183 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [----------] 1 test from neocls_handler_error (2345 ms total) 2024-09-16T08:47:31.183 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: 2024-09-16T08:47:31.183 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [----------] Global test environment tear-down 2024-09-16T08:47:31.183 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [==========] 1 test from 1 test suite ran. (2345 ms total) 2024-09-16T08:47:31.183 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [ PASSED ] 1 test. 2024-09-16T08:47:31.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-09-16T08:47:31.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-09-16T08:47:31.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-09-16T08:47:31.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-09-16T08:47:31.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-09-16T08:47:31.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-09-16T08:47:31.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-09-16T08:47:31.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (412 ms) 2024-09-16T08:47:31.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-09-16T08:47:31.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (9 ms) 2024-09-16T08:47:31.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-09-16T08:47:31.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (10 ms) 2024-09-16T08:47:31.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-09-16T08:47:31.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (28 ms) 2024-09-16T08:47:31.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-09-16T08:47:31.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (9 ms) 2024-09-16T08:47:31.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-09-16T08:47:31.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (9 ms) 2024-09-16T08:47:31.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-09-16T08:47:31.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (9 ms) 2024-09-16T08:47:31.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-09-16T08:47:31.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (6 ms) 2024-09-16T08:47:31.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-09-16T08:47:31.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (4 ms) 2024-09-16T08:47:31.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-09-16T08:47:31.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (5 ms) 2024-09-16T08:47:31.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-09-16T08:47:31.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (6 ms) 2024-09-16T08:47:31.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-09-16T08:47:31.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (5 ms) 2024-09-16T08:47:31.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-09-16T08:47:31.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (13 ms) 2024-09-16T08:47:31.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-09-16T08:47:31.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (11 ms) 2024-09-16T08:47:31.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-09-16T08:47:31.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (10 ms) 2024-09-16T08:47:31.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-09-16T08:47:31.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (6 ms) 2024-09-16T08:47:31.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (552 ms total) 2024-09-16T08:47:31.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: 2024-09-16T08:47:31.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-09-16T08:47:31.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (2404 ms total) 2024-09-16T08:47:31.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-09-16T08:47:31.303 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: Running main() from gmock_main.cc 2024-09-16T08:47:31.303 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-09-16T08:47:31.304 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [----------] Global test environment set-up. 2024-09-16T08:47:31.304 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-09-16T08:47:31.304 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-09-16T08:47:31.304 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (61 ms) 2024-09-16T08:47:31.304 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-09-16T08:47:31.304 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (75 ms) 2024-09-16T08:47:31.304 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-09-16T08:47:31.304 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (2334 ms) 2024-09-16T08:47:31.304 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-09-16T08:47:31.304 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.136623+0000 mon.a [INF] from='client.24748 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi160-76059-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.304 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.136697+0000 mon.a [INF] from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi160-75783-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.304 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.136746+0000 mon.a [INF] from='client.? v1:172.21.15.160:0/1702248129' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.305 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.136782+0000 mon.a [INF] from='client.24772 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi160-76301-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.305 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.136823+0000 mon.a [INF] from='client.24724 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi160-75849-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.305 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.136862+0000 mon.a [INF] from='client.24694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi160-75757-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.305 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.136901+0000 mon.a [INF] from='client.34697 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi160-75827-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.305 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.136951+0000 mon.a [INF] from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi160-75791-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.305 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.137006+0000 mon.a [INF] from='client.34739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi160-76133-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.305 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.137054+0000 mon.a [INF] from='client.24730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi160-75864-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.305 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.137092+0000 mon.a [INF] from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi160-76079-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.305 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.137138+0000 mon.a [INF] from='client.24871 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76320-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.24748 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi160-76059-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi160-75783-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1702248129' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.24772 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi160-76301-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.24724 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi160-75849-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.24694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi160-75757-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.34697 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi160-75827-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi160-75791-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.34739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi160-76133-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.24730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi160-75864-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi160-76079-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.24871 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76320-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi160-75741-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.34757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi160-76254-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.24751 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.24760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi160-76142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.24820 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi160-76516-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.24868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76376-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T08:47:31.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:31.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"} : dispatch 2024-09-16T08:47:31.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[30137]: from='client.24730 ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"} : dispatch 2024-09-16T08:47:31.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.24748 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi160-76059-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi160-75783-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1702248129' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.24772 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi160-76301-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.24724 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi160-75849-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.24694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi160-75757-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.34697 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi160-75827-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi160-75791-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.34739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi160-76133-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.24730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi160-75864-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi160-76079-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.24871 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76320-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi160-75741-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.34757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi160-76254-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.24751 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.24760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi160-76142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.24820 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi160-76516-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.24868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76376-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T08:47:31.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:31.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"} : dispatch 2024-09-16T08:47:31.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:31 smithi160 ceph-mon[40028]: from='client.24730 ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"} : dispatch 2024-09-16T08:47:31.554 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.137184+0000 mon.a [INF] from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application en********* 2024-09-16T08:47:31.554 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_9_[77452]: starting. 2024-09-16T08:47:31.554 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_9_[77452]: creating pool ceph_test_rados_list_parallel.smithi160-76567 2024-09-16T08:47:31.554 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_9_[77452]: created object 0... 2024-09-16T08:47:31.554 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_9_[77452]: created object 25... 2024-09-16T08:47:31.554 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_9_[77452]: created object 49... 2024-09-16T08:47:31.554 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_9_[77452]: finishing. 2024-09-16T08:47:31.554 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_9_[77452]: shutting down. 2024-09-16T08:47:31.555 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.555 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.555 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.555 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_10_[77453]: starting. 2024-09-16T08:47:31.555 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_10_[77453]: listing objects. 2024-09-16T08:47:31.555 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_10_[77453]: listed object 0... 2024-09-16T08:47:31.555 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_10_[77453]: listed object 25... 2024-09-16T08:47:31.555 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_10_[77453]: listed object 50... 2024-09-16T08:47:31.555 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_10_[77453]: listed object 75... 2024-09-16T08:47:31.555 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_10_[77453]: saw 98 objects 2024-09-16T08:47:31.555 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_10_[77453]: shutting down. 2024-09-16T08:47:31.556 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.556 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.556 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.556 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_13_[77456]: starting. 2024-09-16T08:47:31.556 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_13_[77456]: removed 25 objects... 2024-09-16T08:47:31.556 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_13_[77456]: removed half of the objects 2024-09-16T08:47:31.556 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_13_[77456]: removed 50 objects... 2024-09-16T08:47:31.556 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_13_[77456]: removed 50 objects 2024-09-16T08:47:31.556 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_13_[77456]: shutting down. 2024-09-16T08:47:31.557 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.557 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.557 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.557 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_11_[77454]: starting. 2024-09-16T08:47:31.557 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_11_[77454]: added 25 objects... 2024-09-16T08:47:31.557 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_11_[77454]: added half of the objects 2024-09-16T08:47:31.557 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_11_[77454]: added 50 objects... 2024-09-16T08:47:31.557 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_11_[77454]: added 50 objects 2024-09-16T08:47:31.557 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_11_[77454]: shutting down. 2024-09-16T08:47:31.557 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.557 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.558 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.558 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_12_[77455]: starting. 2024-09-16T08:47:31.558 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_12_[77455]: added 25 objects... 2024-09-16T08:47:31.558 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_12_[77455]: added half of the objects 2024-09-16T08:47:31.558 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_12_[77455]: added 50 objects... 2024-09-16T08:47:31.558 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_12_[77455]: added 50 objects 2024-09-16T08:47:31.558 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_12_[77455]: shutting down. 2024-09-16T08:47:31.558 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.558 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.558 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.558 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.559 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_14_[77834]: starting. 2024-09-16T08:47:31.559 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_14_[77834]: creating pool ceph_test_rados_list_parallel.smithi160-76567 2024-09-16T08:47:31.559 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_14_[77834]: created object 0... 2024-09-16T08:47:31.559 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_14_[77834]: created object 25... 2024-09-16T08:47:31.559 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_14_[77834]: created object 49... 2024-09-16T08:47:31.559 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_14_[77834]: finishing. 2024-09-16T08:47:31.559 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_14_[77834]: shutting down. 2024-09-16T08:47:31.559 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.559 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.559 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.559 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:31.560 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[77835]: starting. 2024-09-16T08:47:31.560 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[77835]: listing objects. 2024-09-16T08:47:31.560 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[77835]: listed object 0... 2024-09-16T08:47:31.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.24748 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi160-76059-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi160-75783-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1702248129' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.24772 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi160-76301-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.24724 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi160-75849-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.24694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi160-75757-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.34697 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi160-75827-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi160-75791-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.34739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi160-76133-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.24730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi160-75864-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi160-76079-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.24871 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76320-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi160-75741-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.34757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi160-76254-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.24751 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.24760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi160-76142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.24820 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi160-76516-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.24868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76376-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:31.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T08:47:31.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:31.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"} : dispatch 2024-09-16T08:47:31.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:31 smithi169 ceph-mon[33465]: from='client.24730 ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"} : dispatch 2024-09-16T08:47:32.153 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[77835]: listed object 2able","pool": "TooBig_smithi160-75741-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:32.153 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.137226+0000 mon.a [INF] from='client.34757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi160-76254-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:32.153 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.137267+0000 mon.a [INF] from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:32.153 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.137308+0000 mon.a [INF] from='client.24751 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:32.153 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.137356+0000 mon.a [INF] from='client.24760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi160-76142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:32.153 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.137408+0000 mon.a [INF] from='client.24820 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi160-76516-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:32.153 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:30.137459+0000 mon.a [INF] from='client.24868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76376-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:32.153 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.038152+0000 mon.b [INF] from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"} : dispatch 2024-09-16T08:47:32.153 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.038463+0000 mon.a [INF] from='client.24730 ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"} : dispatch 2024-09-16T08:47:32.153 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.136683+0000 mon.a [WRN] Health check failed: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:47:32.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.147091+0000 mon.a [INF] from='client.24898 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi160-76956-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi160-76956-1"}]': finished 2024-09-16T08:47:32.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.147206+0000 mon.a [INF] from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi160-77061-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi160-77061-1"}]': finished 2024-09-16T08:47:32.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.147274+0000 mon.a [INF] from='client.34769 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi160-76317-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"}]': finished 2024-09-16T08:47:32.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.147352+0000 mon.a [INF] from='client.24730 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"}]': finished 2024-09-16T08:47:32.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.174382+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:32.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.175241+0000 mon.a [INF] from='client.34952 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:32.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.289387+0000 mon.b [INF] from='client.? v1:172.21.15.160:0/3285837583' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-16T08:47:32.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.292414+0000 client.admin [INF] onexx 2024-09-16T08:47:32.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.299383+0000 mon.b [INF] from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:32.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.311098+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:32.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.319214+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:32.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.326316+0000 mon.a [INF] from='client.25099 ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-16T08:47:32.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.326508+0000 mon.a [INF] from='client.35093 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:32.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.330659+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:32.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.338589+0000 mon.a [INF] from='client.35090 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:32.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.338895+0000 mon.a [INF] from='client.25111 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:32.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.343387+0000 mon.b [INF] from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:32.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.343485+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:32.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.346277+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:32.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.355931+0000 mon.a [INF] from='client.35093 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:32.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.358822+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.368247+0000 mon.a [INF] from='client.35090 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:32.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.373759+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi160-76254-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.181 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.375440+0000 mon.a [INF] from='client.35102 ' entity='client.admin' cmd={"prefix": "osd erasure-cod5... 2024-09-16T08:47:32.181 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[77835]: listed object 50... 2024-09-16T08:47:32.181 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[77835]: listed object 75... 2024-09-16T08:47:32.181 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[77835]: listed object 100... 2024-09-16T08:47:32.181 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[77835]: listed object 125... 2024-09-16T08:47:32.181 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[77835]: saw 150 objects 2024-09-16T08:47:32.181 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[77835]: shutting down. 2024-09-16T08:47:32.181 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:32.182 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:32.182 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:32.182 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:32.182 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_16_[77836]: starting. 2024-09-16T08:47:32.182 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_16_[77836]: added 25 objects... 2024-09-16T08:47:32.182 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_16_[77836]: added half of the objects 2024-09-16T08:47:32.182 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_16_[77836]: added 50 objects... 2024-09-16T08:47:32.182 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_16_[77836]: added 50 objects 2024-09-16T08:47:32.182 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_16_[77836]: shutting down. 2024-09-16T08:47:32.182 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:32.182 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:32.182 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:32.182 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:32.182 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-16T08:47:32.183 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******* SUCCESS ********** 2024-09-16T08:47:32.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: pgmap v51: 1124 pgs: 992 unknown, 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:47:32.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: Health check failed: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:47:32.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.24898 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi160-76956-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi160-76956-1"}]': finished 2024-09-16T08:47:32.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi160-77061-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi160-77061-1"}]': finished 2024-09-16T08:47:32.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.34769 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi160-76317-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"}]': finished 2024-09-16T08:47:32.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.24730 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"}]': finished 2024-09-16T08:47:32.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:32.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T08:47:32.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:32.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:32.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3285837583' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-16T08:47:32.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: onexx 2024-09-16T08:47:32.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:32.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:32.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:32.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.25099 ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-16T08:47:32.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:32.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:32.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:32.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:32.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: pgmap v51: 1124 pgs: 992 unknown, 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:47:32.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: Health check failed: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:47:32.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.24898 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi160-76956-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi160-76956-1"}]': finished 2024-09-16T08:47:32.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi160-77061-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi160-77061-1"}]': finished 2024-09-16T08:47:32.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.34769 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi160-76317-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"}]': finished 2024-09-16T08:47:32.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.24730 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"}]': finished 2024-09-16T08:47:32.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:32.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T08:47:32.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:32.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:32.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3285837583' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-16T08:47:32.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: onexx 2024-09-16T08:47:32.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:32.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:32.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:32.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.25099 ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-16T08:47:32.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:32.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:32.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:32.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:32.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:32.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:32.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:32.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:32.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:32.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi160-76254-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:32.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:32.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi160-76254-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:32.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:32.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi160-75783-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[30137]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi160-75783-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:32.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:32.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:32.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:32.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:32.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi160-76254-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:32.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:32.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi160-76254-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:32.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:32.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi160-75783-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:32 smithi160 ceph-mon[40028]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi160-75783-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.501 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: pgmap v51: 1124 pgs: 992 unknown, 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:47:32.501 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: Health check failed: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:47:32.501 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.24898 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi160-76956-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi160-76956-1"}]': finished 2024-09-16T08:47:32.501 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi160-77061-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi160-77061-1"}]': finished 2024-09-16T08:47:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.34769 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi160-76317-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"}]': finished 2024-09-16T08:47:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.24730 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"}]': finished 2024-09-16T08:47:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T08:47:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3285837583' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-16T08:47:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: onexx 2024-09-16T08:47:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.25099 ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-16T08:47:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:32.503 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:32.503 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:32.503 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:32.503 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:32.503 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:32.503 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:32.503 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.503 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:32.503 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi160-76254-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.503 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:32.503 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:32.503 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.503 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi160-76254-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.504 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.504 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:32.504 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.504 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:32.504 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi160-75783-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:32.504 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:32 smithi169 ceph-mon[33465]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi160-75783-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:33.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.25099 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-16T08:47:33.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3285837583' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-16T08:47:33.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:33.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: twoxx 2024-09-16T08:47:33.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.25099 ' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-16T08:47:33.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:33.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.35093 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:33.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.35090 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi160-76254-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:33.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:33.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.35102 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi160-75783-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:33.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T08:47:33.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:33.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4160421474' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi160-75741-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/330058133' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/207482417' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:33.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.25060 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi160-75741-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.25129 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.25099 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-16T08:47:33.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3285837583' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-16T08:47:33.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:33.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: twoxx 2024-09-16T08:47:33.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.25099 ' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-16T08:47:33.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:33.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.35093 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:33.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.35090 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi160-76254-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:33.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:33.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.35102 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi160-75783-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:33.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T08:47:33.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:33.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4160421474' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi160-75741-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/330058133' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/207482417' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:33.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.25060 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi160-75741-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.25129 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi160-75783-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:33.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3189408078' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2315144845' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:33.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.25069 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi160-75783-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:33.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi160-76142-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:33.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi160-76254-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:33.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi160-76142-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:33.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi160-76254-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:33.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.25105 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.24898 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:33.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.0"} : dispatch 2024-09-16T08:47:33.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.1"} : dispatch 2024-09-16T08:47:33.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.2"} : dispatch 2024-09-16T08:47:33.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.3"} : dispatch 2024-09-16T08:47:33.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.4"} : dispatch 2024-09-16T08:47:33.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.5"} : dispatch 2024-09-16T08:47:33.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.6"} : dispatch 2024-09-16T08:47:33.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.7"} : dispatch 2024-09-16T08:47:33.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.8"} : dispatch 2024-09-16T08:47:33.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.9"} : dispatch 2024-09-16T08:47:33.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi160-75783-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:33.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3189408078' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2315144845' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:33.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.25069 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi160-75783-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:33.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi160-76142-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:33.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi160-76254-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:33.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi160-76142-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:33.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi160-76254-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:33.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.25105 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.24898 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:33.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.0"} : dispatch 2024-09-16T08:47:33.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.1"} : dispatch 2024-09-16T08:47:33.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.2"} : dispatch 2024-09-16T08:47:33.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.3"} : dispatch 2024-09-16T08:47:33.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.4"} : dispatch 2024-09-16T08:47:33.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.5"} : dispatch 2024-09-16T08:47:33.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.6"} : dispatch 2024-09-16T08:47:33.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.7"} : dispatch 2024-09-16T08:47:33.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.8"} : dispatch 2024-09-16T08:47:33.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.9"} : dispatch 2024-09-16T08:47:33.524 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.25099 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-16T08:47:33.524 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3285837583' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-16T08:47:33.524 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:33.524 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: twoxx 2024-09-16T08:47:33.525 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.25099 ' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-16T08:47:33.525 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:33.525 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.35093 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:33.525 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.35090 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi160-76254-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:33.525 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:33.525 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.35102 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi160-75783-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:33.525 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T08:47:33.525 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:33.525 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4160421474' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi160-75741-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.525 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/330058133' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.525 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/207482417' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.526 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:33.526 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.25060 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi160-75741-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.526 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.25129 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.526 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi160-75783-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:33.526 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3189408078' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.526 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2315144845' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:33.527 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.25069 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.527 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi160-75783-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:33.527 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi160-76142-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:33.527 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi160-76254-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:33.527 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi160-76142-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:33.527 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi160-76254-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:33.527 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.25105 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:33.527 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.24898 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:33.527 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.0"} : dispatch 2024-09-16T08:47:33.527 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.1"} : dispatch 2024-09-16T08:47:33.528 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.2"} : dispatch 2024-09-16T08:47:33.528 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.3"} : dispatch 2024-09-16T08:47:33.528 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.4"} : dispatch 2024-09-16T08:47:33.528 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.5"} : dispatch 2024-09-16T08:47:33.528 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.6"} : dispatch 2024-09-16T08:47:33.528 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.7"} : dispatch 2024-09-16T08:47:33.528 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.8"} : dispatch 2024-09-16T08:47:33.528 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.9"} : dispatch 2024-09-16T08:47:34.194 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_1_[76794]: starting. 2024-09-16T08:47:34.195 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_1_[76794]: creating pool ceph_test_rados_delete_pools_parallel.smithi160-76649 2024-09-16T08:47:34.195 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_1_[76794]: created object 0... 2024-09-16T08:47:34.195 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_1_[76794]: created object 25... 2024-09-16T08:47:34.195 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_1_[76794]: created object 49... 2024-09-16T08:47:34.195 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_1_[76794]: finishing. 2024-09-16T08:47:34.195 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_1_[76794]: shutting down. 2024-09-16T08:47:34.195 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_2_[76796]: starting. 2024-09-16T08:47:34.195 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_2_[76796]: deleting pool ceph_test_rados_delete_pools_parallel.smithi160-76649 2024-09-16T08:47:34.195 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_2_[76796]: shutting down. 2024-09-16T08:47:34.195 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: ******************************* 2024-09-16T08:47:34.195 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_3_[77534]: starting. 2024-09-16T08:47:34.196 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_3_[77534]: creating pool ceph_test_rados_delete_pools_parallel.smithi160-76649 2024-09-16T08:47:34.196 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_3_[77534]: created object 0... 2024-09-16T08:47:34.196 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_3_[77534]: created object 25... 2024-09-16T08:47:34.196 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_3_[77534]: created object 49... 2024-09-16T08:47:34.196 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_3_[77534]: finishing. 2024-09-16T08:47:34.196 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_3_[77534]: shutting down. 2024-09-16T08:47:34.196 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: ******************************* 2024-09-16T08:47:34.196 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_5_[77536]: starting. 2024-09-16T08:47:34.196 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_5_[77536]: listing objects. 2024-09-16T08:47:34.196 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_5_[77536]: listed object 0... 2024-09-16T08:47:34.196 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_5_[77536]: listed object 25... 2024-09-16T08:47:34.196 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_5_[77536]: saw 50 objects 2024-09-16T08:47:34.196 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_5_[77536]: shutting down. 2024-09-16T08:47:34.196 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: ******************************* 2024-09-16T08:47:34.197 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_4_[77535]: starting. 2024-09-16T08:47:34.197 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_4_[77535]: deleting pool ceph_test_rados_delete_pools_parallel.smithi160-76649 2024-09-16T08:47:34.197 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_4_[77535]: shutting down. 2024-09-16T08:47:34.197 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: ******************************* 2024-09-16T08:47:34.197 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: ******************************* 2024-09-16T08:47:34.197 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-09-16T08:47:34.198 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_1_[76697]: starting. 2024-09-16T08:47:34.199 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_1_[76697]: creating pool ceph_test_rados_open_pools_parallel.smithi160-76607 2024-09-16T08:47:34.199 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_1_[76697]: created object 0... 2024-09-16T08:47:34.199 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_1_[76697]: created object 25... 2024-09-16T08:47:34.199 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_1_[76697]: created object 49... 2024-09-16T08:47:34.199 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_1_[76697]: finishing. 2024-09-16T08:47:34.199 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_1_[76697]: shutting down. 2024-09-16T08:47:34.199 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_2_[76700]: starting. 2024-09-16T08:47:34.199 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_2_[76700]: rados_pool_create. 2024-09-16T08:47:34.199 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_2_[76700]: rados_ioctx_create. 2024-09-16T08:47:34.199 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_2_[76700]: shutting down. 2024-09-16T08:47:34.199 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: ******************************* 2024-09-16T08:47:34.199 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_3_[77532]: starting. 2024-09-16T08:47:34.199 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_3_[77532]: creating pool ceph_test_rados_open_pools_parallel.smithi160-76607 2024-09-16T08:47:34.199 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_3_[77532]: created object 0... 2024-09-16T08:47:34.200 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_3_[77532]: created object 25... 2024-09-16T08:47:34.200 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_3_[77532]: created object 49... 2024-09-16T08:47:34.200 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_3_[77532]: finishing. 2024-09-16T08:47:34.200 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_3_[77532]: shutting down. 2024-09-16T08:47:34.200 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: ******************************* 2024-09-16T08:47:34.200 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_4_[77533]: starting. 2024-09-16T08:47:34.200 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_4_[77533]: rados_pool_create. 2024-09-16T08:47:34.200 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_4_[77533]: rados_ioctx_create. 2024-09-16T08:47:34.200 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_4_[77533]: shutting down. 2024-09-16T08:47:34.200 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: ******************************* 2024-09-16T08:47:34.200 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: ******************************* 2024-09-16T08:47:34.201 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-09-16T08:47:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: pgmap v54: 852 pgs: 720 unknown, 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:47:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-16T08:47:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-16T08:47:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-16T08:47:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-16T08:47:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-16T08:47:34.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-16T08:47:34.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-16T08:47:34.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-16T08:47:34.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-16T08:47:34.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-16T08:47:34.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.25099 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-16T08:47:34.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:34.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.25060 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi160-75741-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:34.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.25129 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:34.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:34.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.25105 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:34.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.24898 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi160-76956-1"}]': finished 2024-09-16T08:47:34.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2315144845' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:34.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T08:47:34.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.24898 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:34.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/985108347' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:34.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:34.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: pgmap v54: 852 pgs: 720 unknown, 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:47:34.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-16T08:47:34.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-16T08:47:34.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-16T08:47:34.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-16T08:47:34.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-16T08:47:34.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-16T08:47:34.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-16T08:47:34.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-16T08:47:34.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-16T08:47:34.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-16T08:47:34.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.25099 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-16T08:47:34.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:34.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.25060 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi160-75741-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:34.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.25129 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:34.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:34.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.25105 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:34.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.24898 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi160-76956-1"}]': finished 2024-09-16T08:47:34.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2315144845' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:34.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T08:47:34.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.24898 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:34.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/985108347' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:34.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:34.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:34.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:34.454 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:34.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.24913 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:34.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: 17.9 deep-scrub starts 2024-09-16T08:47:34.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:34.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:34.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:34.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:34.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:34.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:34.455 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:34.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:34.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:34.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:34.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi160-75827-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:34.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi160-75827-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:34.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: 17.9 deep-scrub ok 2024-09-16T08:47:34.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:34.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:47:34.456 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: 17.2 deep-scrub starts 2024-09-16T08:47:34.457 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: 17.1 deep-scrub starts 2024-09-16T08:47:34.457 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: 17.1 deep-scrub ok 2024-09-16T08:47:34.457 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: 17.0 deep-scrub starts 2024-09-16T08:47:34.457 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: 17.0 deep-scrub ok 2024-09-16T08:47:34.457 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2420422157' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-16T08:47:34.457 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:34.457 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:34.457 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:34.457 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.24913 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:34.458 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: 17.9 deep-scrub starts 2024-09-16T08:47:34.458 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:34.458 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:34.458 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:34.458 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:34.458 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:34.458 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:34.458 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:34.458 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:34.458 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:34.458 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:34.458 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi160-75827-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:34.458 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi160-75827-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:34.459 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: 17.9 deep-scrub ok 2024-09-16T08:47:34.459 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:34.459 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:47:34.459 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: 17.2 deep-scrub starts 2024-09-16T08:47:34.459 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: 17.1 deep-scrub starts 2024-09-16T08:47:34.459 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: 17.1 deep-scrub ok 2024-09-16T08:47:34.459 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: 17.0 deep-scrub starts 2024-09-16T08:47:34.459 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: 17.0 deep-scrub ok 2024-09-16T08:47:34.459 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2420422157' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-16T08:47:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: pgmap v54: 852 pgs: 720 unknown, 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:47:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-16T08:47:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-16T08:47:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-16T08:47:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-16T08:47:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-16T08:47:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-16T08:47:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-16T08:47:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-16T08:47:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-16T08:47:34.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-16T08:47:34.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.25099 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-16T08:47:34.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:34.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.25060 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi160-75741-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:34.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.25129 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:34.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:34.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.25105 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:34.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.24898 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi160-76956-1"}]': finished 2024-09-16T08:47:34.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2315144845' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:34.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T08:47:34.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.24898 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi160-76956-1"} : dispatch 2024-09-16T08:47:34.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/985108347' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:34.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:34.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:34.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:34.604 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:34.604 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.24913 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:34.604 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: 17.9 deep-scrub starts 2024-09-16T08:47:34.604 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:34.604 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:34.604 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:34.604 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:34.604 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:34.605 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:34.605 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:34.605 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:34.605 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:34.605 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:34.605 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi160-75827-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:34.605 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi160-75827-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:34.605 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: 17.9 deep-scrub ok 2024-09-16T08:47:34.606 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:34.606 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:47:34.606 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: 17.2 deep-scrub starts 2024-09-16T08:47:34.606 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: 17.1 deep-scrub starts 2024-09-16T08:47:34.606 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: 17.1 deep-scrub ok 2024-09-16T08:47:34.606 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: 17.0 deep-scrub starts 2024-09-16T08:47:34.606 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: 17.0 deep-scrub ok 2024-09-16T08:47:34.606 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2420422157' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-16T08:47:35.099 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:47:34 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:47:35.160 INFO:tasks.workunit.client.0.smithi160.stdout:e-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:35.161 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.375774+0000 mon.a [INF] from='client.25111 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:35.161 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.376044+0000 mon.a [INF] from='client.35093 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:35.161 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.376393+0000 mon.a [INF] from='client.35090 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi160-76254-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:35.161 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.377869+0000 mon.b [INF] from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:35.161 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.377885+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:35.161 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.378368+0000 mon.a [INF] from='client.25111 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:35.161 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.378762+0000 mon.a [INF] from='client.35102 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:35.161 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.385301+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi160-75783-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:35.161 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:31.385918+0000 mon.a [INF] from='client.35102 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi160-75783-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:35.161 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.173540+0000 mon.a [INF] from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:35.161 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.173608+0000 mon.a [INF] from='client.35102 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi160-75783-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi160-75783-16"}]': finished 2024-09-16T08:47:35.161 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.173656+0000 mon.a [INF] from='client.35093 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi160-76142-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi160-76142-7"}]': finished 2024-09-16T08:47:35.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.173718+0000 mon.a [INF] from='client.35090 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi160-76254-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi160-76254-7"}]': finished 2024-09-16T08:47:35.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.173763+0000 mon.a [INF] from='client.24898 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi160-76956-1"}]': finished 2024-09-16T08:47:35.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.173821+0000 mon.a [INF] from='client.34769 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"}]': finished 2024-09-16T08:47:35.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.173857+0000 mon.a [INF] from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:35.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.173911+0000 mon.a [INF] from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi160-77061-1"}]': finished 2024-09-16T08:47:35.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.173965+0000 mon.a [INF] from='client.35150 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:35.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.174005+0000 mon.a [INF] from='client.25204 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi160-75827-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:35.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.179141+0000 mon.b [INF] from='client.? v1:172.21.15.160:0/985108347' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:35.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.180262+0000 mon.b [INF] from='client.? v1:172.21.15.160:0/330058133' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:35.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.182411+0000 mon.b [INF] from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi160-75827-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:35.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.183566+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi160-75849-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:35.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.183777+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:35.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.191012+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:35.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.200623+0000 mon.a [INF] from='client.24913 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:35.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.200990+0000 mon.a [INF] from='client.35150 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi160-75849-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:35.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.201707+0000 mon.a [INF] from='client.25129 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:35.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.201957+0000 mon.a [INF] from='client.34769 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:35.226 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-1 cmd: Running main() from gmock_main.cc 2024-09-16T08:47:35.226 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-09-16T08:47:35.226 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [----------] Global test environment set-up. 2024-09-16T08:47:35.226 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-09-16T08:47:35.226 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-09-16T08:47:35.226 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (1284 ms) 2024-09-16T08:47:35.226 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-09-16T08:47:35.226 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (2005 ms) 2024-09-16T08:47:35.226 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-09-16T08:47:35.226 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (3048 ms) 2024-09-16T08:47:35.226 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [----------] 3 tests from NeoRadosCmd (6337 ms total) 2024-09-16T08:47:35.226 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: 2024-09-16T08:47:35.227 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [----------] Global test environment tear-down 2024-09-16T08:47:35.227 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [==========] 3 tests from 1 test suite ran. (6337 ms total) 2024-09-16T08:47:35.227 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [ PASSED ] 3 tests. 2024-09-16T08:47:35.329 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: Running main() from gmock_main.cc 2024-09-16T08:47:35.329 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [==========] Running 40 tests from 2 test suites. 2024-09-16T08:47:35.329 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [----------] Global test environment set-up. 2024-09-16T08:47:35.329 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [----------] 22 tests from LibRadosIoPP 2024-09-16T08:47:35.329 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: seed 75791 2024-09-16T08:47:35.329 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-09-16T08:47:35.329 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-09-16T08:47:35.329 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-09-16T08:47:35.329 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (376 ms) 2024-09-16T08:47:35.329 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-09-16T08:47:35.330 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (10 ms) 2024-09-16T08:47:35.330 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-09-16T08:47:35.330 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (6 ms) 2024-09-16T08:47:35.330 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadExtentArrayOpPP 2024-09-16T08:47:35.330 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadExtentArrayOpPP (340 ms) 2024-09-16T08:47:35.330 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-09-16T08:47:35.330 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (3 ms) 2024-09-16T08:47:35.330 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-09-16T08:47:35.330 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (2 ms) 2024-09-16T08:47:35.330 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-09-16T08:47:35.330 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (2 ms) 2024-09-16T08:47:35.330 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-09-16T08:47:35.330 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (3 ms) 2024-09-16T08:47:35.330 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-09-16T08:47:35.331 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (3 ms) 2024-09-16T08:47:35.331 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-09-16T08:47:35.331 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (3 ms) 2024-09-16T08:47:35.331 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-09-16T08:47:35.331 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (2 ms) 2024-09-16T08:47:35.331 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-09-16T08:47:35.331 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (5 ms) 2024-09-16T08:47:35.331 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-09-16T08:47:35.331 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (3 ms) 2024-09-16T08:47:35.331 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-09-16T08:47:35.331 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (3 ms) 2024-09-16T08:47:35.331 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-09-16T08:47:35.331 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (4 ms) 2024-09-16T08:47:35.331 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-09-16T08:47:35.331 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (7 ms) 2024-09-16T08:47:35.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-16T08:47:35.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (4 ms) 2024-09-16T08:47:35.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CrcZeroWrite 2024-09-16T08:47:35.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.CrcZeroWrite (3 ms) 2024-09-16T08:47:35.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-09-16T08:47:35.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (5 ms) 2024-09-16T08:47:35.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-09-16T08:47:35.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (5 ms) 2024-09-16T08:47:35.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-09-16T08:47:35.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (5 ms) 2024-09-16T08:47:35.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [----------] 22 tests from LibRadosIoPP (794 ms total) 2024-09-16T08:47:35.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: 2024-09-16T08:47:35.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [----------] 18 tests from LibRadosIoECPP 2024-09-16T08:47:35.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-09-16T08:47:35.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1136 ms) 2024-09-16T08:47:35.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-09-16T08:47:35.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (9 ms) 2024-09-16T08:47:35.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-09-16T08:47:35.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (4 ms) 2024-09-16T08:47:35.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:47:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:47:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: 17.2 deep-scrub ok 2024-09-16T08:47:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: 17.5 deep-scrub starts 2024-09-16T08:47:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: 17.5 deep-scrub ok 2024-09-16T08:47:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.35102 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi160-75783-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi160-75783-16"}]': finished 2024-09-16T08:47:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.35093 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi160-76142-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi160-76142-7"}]': finished 2024-09-16T08:47:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.35090 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi160-76254-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi160-76254-7"}]': finished 2024-09-16T08:47:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.24898 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi160-76956-1"}]': finished 2024-09-16T08:47:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.34769 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"}]': finished 2024-09-16T08:47:35.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:35.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi160-77061-1"}]': finished 2024-09-16T08:47:35.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.35150 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:35.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.25204 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi160-75827-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:35.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T08:47:35.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/985108347' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:35.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/330058133' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:35.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi160-75827-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:35.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi160-75849-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:35.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: 17.2 deep-scrub ok 2024-09-16T08:47:35.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: 17.5 deep-scrub starts 2024-09-16T08:47:35.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: 17.5 deep-scrub ok 2024-09-16T08:47:35.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:35.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:35.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.35102 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi160-75783-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi160-75783-16"}]': finished 2024-09-16T08:47:35.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.35093 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi160-76142-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi160-76142-7"}]': finished 2024-09-16T08:47:35.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.35090 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi160-76254-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi160-76254-7"}]': finished 2024-09-16T08:47:35.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.24898 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi160-76956-1"}]': finished 2024-09-16T08:47:35.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.34769 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"}]': finished 2024-09-16T08:47:35.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:35.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi160-77061-1"}]': finished 2024-09-16T08:47:35.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.35150 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:35.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.25204 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi160-75827-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:35.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T08:47:35.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/985108347' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:35.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/330058133' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:35.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi160-75827-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:35.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi160-75849-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:35.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:35.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:35.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.24913 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:35.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi160-75849-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:35.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.25129 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:35.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:35.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:35.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi160-75827-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:35.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1131943809' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi160-76956-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:35.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: from='client.25381 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi160-76956-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:35.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: 17.3 deep-scrub starts 2024-09-16T08:47:35.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: 17.3 deep-scrub ok 2024-09-16T08:47:35.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: 17.4 deep-scrub starts 2024-09-16T08:47:35.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: 17.4 deep-scrub ok 2024-09-16T08:47:35.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: 17.8 deep-scrub starts 2024-09-16T08:47:35.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[40028]: 17.8 deep-scrub ok 2024-09-16T08:47:35.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:35.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:35.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.24913 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:35.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi160-75849-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:35.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.25129 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:35.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:35.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:35.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi160-75827-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:35.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1131943809' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi160-76956-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:35.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: from='client.25381 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi160-76956-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:35.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: 17.3 deep-scrub starts 2024-09-16T08:47:35.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: 17.3 deep-scrub ok 2024-09-16T08:47:35.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: 17.4 deep-scrub starts 2024-09-16T08:47:35.454 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: 17.4 deep-scrub ok 2024-09-16T08:47:35.454 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: 17.8 deep-scrub starts 2024-09-16T08:47:35.454 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:35 smithi160 ceph-mon[30137]: 17.8 deep-scrub ok 2024-09-16T08:47:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: 17.2 deep-scrub ok 2024-09-16T08:47:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: 17.5 deep-scrub starts 2024-09-16T08:47:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: 17.5 deep-scrub ok 2024-09-16T08:47:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.35102 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi160-75783-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi160-75783-16"}]': finished 2024-09-16T08:47:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.35093 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi160-76142-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi160-76142-7"}]': finished 2024-09-16T08:47:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.35090 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi160-76254-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi160-76254-7"}]': finished 2024-09-16T08:47:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.24898 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi160-76956-1"}]': finished 2024-09-16T08:47:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.34769 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-76317-1"}]': finished 2024-09-16T08:47:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi160-77061-1"}]': finished 2024-09-16T08:47:35.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.35150 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:35.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.25204 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi160-75827-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:35.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T08:47:35.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/985108347' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:35.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/330058133' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:35.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi160-75827-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:35.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi160-75849-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:35.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3819752116' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:35.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:35.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.24913 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi160-77061-1"} : dispatch 2024-09-16T08:47:35.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi160-75849-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:35.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.25129 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:35.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.34769 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"} : dispatch 2024-09-16T08:47:35.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:35.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi160-75827-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:35.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1131943809' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi160-76956-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:35.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: from='client.25381 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi160-76956-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:35.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: 17.3 deep-scrub starts 2024-09-16T08:47:35.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: 17.3 deep-scrub ok 2024-09-16T08:47:35.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: 17.4 deep-scrub starts 2024-09-16T08:47:35.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: 17.4 deep-scrub ok 2024-09-16T08:47:35.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: 17.8 deep-scrub starts 2024-09-16T08:47:35.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:35 smithi169 ceph-mon[33465]: 17.8 deep-scrub ok 2024-09-16T08:47:36.159 INFO:tasks.workunit.client.0.smithi160.stdout: api6T08:47:34.202427+0000 mon.a [INF] from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:36.159 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.202695+0000 mon.a [INF] from='client.25204 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi160-75827-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:36.159 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.257149+0000 mon.b [INF] from='client.? v1:172.21.15.160:0/1131943809' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi160-76956-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:36.159 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:34.289915+0000 mon.a [INF] from='client.25381 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi160-76956-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:36.159 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.161029+0000 mon.b [INF] from='client.? v1:172.21.15.160:0/3285837583' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-16T08:47:36.159 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.161065+0000 client.admin [INF] threexx 2024-09-16T08:47:36.159 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.161315+0000 mon.a [INF] from='client.25099 ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-16T08:47:36.159 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.181983+0000 mon.a [INF] from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi160-77061-1"}]': finished 2024-09-16T08:47:36.160 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.182067+0000 mon.a [INF] from='client.25129 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:36.160 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.182136+0000 mon.a [INF] from='client.34769 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"}]': finished 2024-09-16T08:47:36.160 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.182197+0000 mon.a [INF] from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"}]': finished 2024-09-16T08:47:36.160 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.182264+0000 mon.a [INF] from='client.25381 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi160-76956-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:36.160 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.190036+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-4", "overlaypool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:36.160 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.190069+0000 mon.b [INF] from='client.? v1:172.21.15.160:0/1131943809' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi160-76956-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:36.160 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.191576+0000 mon.b [INF] from='client.? v1:172.21.15.160:0/330058133' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi160-75757-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-16T08:47:36.160 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.211733+0000 mon.a [INF] from='client.25381 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi160-76956-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:36.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: 17.6 deep-scrub starts 2024-09-16T08:47:36.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: 17.6 deep-scrub ok 2024-09-16T08:47:36.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:36.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: pgmap v57: 676 pgs: 96 creating+peering, 128 unknown, 452 active+clean; 145 MiB data, 715 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-09-16T08:47:36.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:36.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3285837583' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-16T08:47:36.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: threexx 2024-09-16T08:47:36.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.25099 ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-16T08:47:36.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi160-77061-1"}]': finished 2024-09-16T08:47:36.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.25129 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:36.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.34769 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"}]': finished 2024-09-16T08:47:36.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"}]': finished 2024-09-16T08:47:36.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.25381 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi160-76956-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:36.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-4", "overlaypool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:36.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1131943809' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi160-76956-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:36.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/330058133' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi160-75757-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-16T08:47:36.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: osdmap e73: 8 total, 8 up, 8 in 2024-09-16T08:47:36.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.25381 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi160-76956-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:36.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/767158736' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-4", "overlaypool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:36.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.25129 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi160-75757-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-16T08:47:36.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: 17.6 deep-scrub starts 2024-09-16T08:47:36.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: 17.6 deep-scrub ok 2024-09-16T08:47:36.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:36.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: pgmap v57: 676 pgs: 96 creating+peering, 128 unknown, 452 active+clean; 145 MiB data, 715 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-09-16T08:47:36.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:36.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3285837583' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-16T08:47:36.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: threexx 2024-09-16T08:47:36.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.25099 ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-16T08:47:36.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi160-77061-1"}]': finished 2024-09-16T08:47:36.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.25129 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:36.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.34769 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"}]': finished 2024-09-16T08:47:36.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"}]': finished 2024-09-16T08:47:36.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.25381 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi160-76956-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:36.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-4", "overlaypool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:36.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1131943809' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi160-76956-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:36.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/330058133' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi160-75757-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-16T08:47:36.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: osdmap e73: 8 total, 8 up, 8 in 2024-09-16T08:47:36.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.25381 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi160-76956-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:36.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/767158736' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-4", "overlaypool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:36.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.25129 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi160-75757-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-16T08:47:36.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4027487533' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.25387 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1832056001' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi160-75741-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.35303 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.35330 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi160-75741-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/841615988' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi160-77061-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:36.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.35348 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi160-77061-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:36.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi160-75791-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-16T08:47:36.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi160-75791-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-16T08:47:36.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3964467320' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-16T08:47:36.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4027487533' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.25387 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1832056001' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi160-75741-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.35303 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.35330 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi160-75741-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/841615988' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi160-77061-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:36.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.35348 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi160-77061-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:36.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi160-75791-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-16T08:47:36.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi160-75791-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-16T08:47:36.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3964467320' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-16T08:47:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: 17.6 deep-scrub starts 2024-09-16T08:47:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: 17.6 deep-scrub ok 2024-09-16T08:47:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: pgmap v57: 676 pgs: 96 creating+peering, 128 unknown, 452 active+clean; 145 MiB data, 715 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-09-16T08:47:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3285837583' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-16T08:47:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: threexx 2024-09-16T08:47:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.25099 ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-16T08:47:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi160-77061-1"}]': finished 2024-09-16T08:47:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.25129 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-75757-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.34769 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-76317-1"}]': finished 2024-09-16T08:47:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"}]': finished 2024-09-16T08:47:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.25381 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi160-76956-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-4", "overlaypool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1131943809' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi160-76956-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/330058133' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi160-75757-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-16T08:47:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: osdmap e73: 8 total, 8 up, 8 in 2024-09-16T08:47:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.25381 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi160-76956-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/767158736' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-4", "overlaypool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.25129 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi160-75757-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-16T08:47:36.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4027487533' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.25387 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1832056001' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi160-75741-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.35303 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.35330 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi160-75741-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:36.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/841615988' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi160-77061-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:36.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.35348 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi160-77061-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:36.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi160-75791-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-16T08:47:36.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi160-75791-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-16T08:47:36.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3964467320' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-16T08:47:37.242 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.2 list: Running main() from gmock_main.cc 2024-09-16T08:47:37.243 INFO:tasks.workunit.client.0.smithi160.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-09-16T08:47:37.243 INFO:tasks.workunit.client.0.smithi160.stdout: list: [----------] Global test environment set-up. 2024-09-16T08:47:37.243 INFO:tasks.workunit.client.0.smithi160.stdout: list: [----------] 3 tests from NeoradosList 2024-09-16T08:47:37.243 INFO:tasks.workunit.client.0.smithi160.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-09-16T08:47:37.243 INFO:tasks.workunit.client.0.smithi160.stdout: list: [ OK ] NeoradosList.ListObjects (2248 ms) 2024-09-16T08:47:37.243 INFO:tasks.workunit.client.0.smithi160.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-09-16T08:47:37.243 INFO:tasks.workunit.client.0.smithi160.stdout: list: [ OK ] NeoradosList.ListObjectsNS (3031 ms) 2024-09-16T08:47:37.243 INFO:tasks.workunit.client.0.smithi160.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-09-16T08:47:37.243 INFO:tasks.workunit.client.0.smithi160.stdout: list: [ OK ] NeoradosList.ListObjectsMany (3042 ms) 2024-09-16T08:47:37.243 INFO:tasks.workunit.client.0.smithi160.stdout: list: [----------] 3 tests from NeoradosList (8321 ms total) 2024-09-16T08:47:37.243 INFO:tasks.workunit.client.0.smithi160.stdout: list: 2024-09-16T08:47:37.243 INFO:tasks.workunit.client.0.smithi160.stdout: list: [----------] Global test environment tear-down 2024-09-16T08:47:37.243 INFO:tasks.workunit.client.0.smithi160.stdout: list: [==========] 3 tests from 1 test suite ran. (8321 ms total) 2024-09-16T08:47:37.243 INFO:tasks.workunit.client.0.smithi160.stdout: list: [ PASSED ] 3 tests. 2024-09-16T08:47:37.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: 17.7 deep-scrub starts 2024-09-16T08:47:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: 17.7 deep-scrub ok 2024-09-16T08:47:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.25099 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-16T08:47:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3285837583' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-16T08:47:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: fourxx 2024-09-16T08:47:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.25099 ' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-16T08:47:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.35150 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi160-75849-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi160-75849-10"}]': finished 2024-09-16T08:47:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.25204 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi160-75827-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi160-75827-10"}]': finished 2024-09-16T08:47:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-4", "overlaypool": "test-rados-api-smithi160-76079-4-cache"}]': finished 2024-09-16T08:47:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.25129 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi160-75757-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-16T08:47:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.25387 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.35303 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:37.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi160-75741-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:37.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.35348 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi160-77061-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:37.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi160-75791-24", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T08:47:37.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: osdmap e74: 8 total, 8 up, 8 in 2024-09-16T08:47:37.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/841615988' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi160-77061-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:37.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-4-cache", "mode": "writeback"} : dispatch 2024-09-16T08:47:37.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.35348 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi160-77061-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:37.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: 17.7 deep-scrub starts 2024-09-16T08:47:37.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: 17.7 deep-scrub ok 2024-09-16T08:47:37.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.25099 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-16T08:47:37.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:37.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3285837583' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-16T08:47:37.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: fourxx 2024-09-16T08:47:37.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.25099 ' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-16T08:47:37.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.35150 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi160-75849-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi160-75849-10"}]': finished 2024-09-16T08:47:37.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.25204 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi160-75827-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi160-75827-10"}]': finished 2024-09-16T08:47:37.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-4", "overlaypool": "test-rados-api-smithi160-76079-4-cache"}]': finished 2024-09-16T08:47:37.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.25129 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi160-75757-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-16T08:47:37.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.25387 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:37.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.35303 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:37.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi160-75741-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:37.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.35348 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi160-77061-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:37.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi160-75791-24", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T08:47:37.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: osdmap e74: 8 total, 8 up, 8 in 2024-09-16T08:47:37.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/841615988' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi160-77061-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:37.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-4-cache", "mode": "writeback"} : dispatch 2024-09-16T08:47:37.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.35348 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi160-77061-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:37.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-4-cache", "mode": "writeback"} : dispatch 2024-09-16T08:47:37.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:37.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:37.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:37.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:37.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:37.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:37.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:37.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[40028]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:37.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-4-cache", "mode": "writeback"} : dispatch 2024-09-16T08:47:37.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:37.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:37.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:37.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:37.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:37.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:37.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:37.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:37 smithi160 ceph-mon[30137]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: 17.7 deep-scrub starts 2024-09-16T08:47:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: 17.7 deep-scrub ok 2024-09-16T08:47:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.25099 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-16T08:47:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3285837583' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-16T08:47:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: fourxx 2024-09-16T08:47:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.25099 ' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-16T08:47:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.35150 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi160-75849-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi160-75849-10"}]': finished 2024-09-16T08:47:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.25204 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi160-75827-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi160-75827-10"}]': finished 2024-09-16T08:47:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-4", "overlaypool": "test-rados-api-smithi160-76079-4-cache"}]': finished 2024-09-16T08:47:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.25129 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi160-75757-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-16T08:47:37.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.25387 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:37.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.35303 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:37.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi160-75741-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:37.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.35348 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi160-77061-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:37.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi160-75791-24", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T08:47:37.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: osdmap e74: 8 total, 8 up, 8 in 2024-09-16T08:47:37.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/841615988' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi160-77061-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:37.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-4-cache", "mode": "writeback"} : dispatch 2024-09-16T08:47:37.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.35348 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi160-77061-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:37.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-4-cache", "mode": "writeback"} : dispatch 2024-09-16T08:47:37.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:37.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:37.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:37.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:37.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:37.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:37.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:37.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:37 smithi169 ceph-mon[33465]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:38.214 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: Running main() from gmock_main.cc 2024-09-16T08:47:38.215 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-09-16T08:47:38.215 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [----------] Global test environment set-up. 2024-09-16T08:47:38.215 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-09-16T08:47:38.215 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-09-16T08:47:38.215 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: handle_notify cookie 94625516487840 notify_id 292057776128 notifier_gid 24934 2024-09-16T08:47:38.215 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (2202 ms) 2024-09-16T08:47:38.215 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-09-16T08:47:38.215 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: Trying... 2024-09-16T08:47:38.215 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: handle_notify cookie 94625528360528 notify_id 304942678016 notifier_gid 25108 2024-09-16T08:47:38.215 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: Waiting for 3.000000000s 2024-09-16T08:47:38.215 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: Timed out. 2024-09-16T08:47:38.215 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: Flushing... 2024-09-16T08:47:38.216 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: Flushed... 2024-09-16T08:47:38.216 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (7046 ms) 2024-09-16T08:47:38.216 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (9248 ms total) 2024-09-16T08:47:38.216 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: 2024-09-16T08:47:38.216 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [----------] Global test environment tear-down 2024-09-16T08:47:38.216 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (9248 ms total) 2024-09-16T08:47:38.216 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [ PASSED ] 2 tests. 2024-09-16T08:47:38.217 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: Running main() from gmock_main.cc 2024-09-16T08:47:38.217 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-09-16T08:47:38.217 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [----------] Global test environment set-up. 2024-09-16T08:47:38.218 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-09-16T08:47:38.218 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-09-16T08:47:38.218 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (407 ms) 2024-09-16T08:47:38.218 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-09-16T08:47:38.218 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-09-16T08:47:38.218 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-09-16T08:47:38.218 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: no timeout :/ 2024-09-16T08:47:38.218 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: no timeout :/ 2024-09-16T08:47:38.218 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: no timeout :/ 2024-09-16T08:47:38.218 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: no timeout :/ 2024-09-16T08:47:38.218 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: no timeout :/ 2024-09-16T08:47:38.218 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (42 ms) 2024-09-16T08:47:38.218 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-09-16T08:47:38.219 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (10 ms) 2024-09-16T08:47:38.219 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-09-16T08:47:38.219 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.Checksum (3 ms) 2024-09-16T08:47:38.219 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-09-16T08:47:38.219 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (6 ms) 2024-09-16T08:47:38.219 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-09-16T08:47:38.219 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (4 ms) 2024-09-16T08:47:38.219 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-09-16T08:47:38.219 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (6 ms) 2024-09-16T08:47:38.219 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-09-16T08:47:38.219 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (1 ms) 2024-09-16T08:47:38.219 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-09-16T08:47:38.219 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.TruncTest (4 ms) 2024-09-16T08:47:38.219 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-09-16T08:47:38.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (4 ms) 2024-09-16T08:47:38.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-09-16T08:47:38.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (5 ms) 2024-09-16T08:47:38.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-09-16T08:47:38.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.RmXattr (12 ms) 2024-09-16T08:47:38.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-09-16T08:47:38.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.XattrIter (7 ms) 2024-09-16T08:47:38.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [----------] 14 tests from LibRadosIo (511 ms total) 2024-09-16T08:47:38.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: 2024-09-16T08:47:38.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-09-16T08:47:38.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-09-16T08:47:38.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1081 ms) 2024-09-16T08:47:38.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-09-16T08:47:38.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (16 ms) 2024-09-16T08:47:38.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-09-16T08:47:38.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (16 ms) 2024-09-16T08:47:38.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-09-16T08:47:38.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (11 ms) 2024-09-16T08:47:38.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-09-16T08:47:38.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (8 ms) 2024-09-16T08:47:38.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-09-16T08:47:38.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (3 ms) 2024-09-16T08:47:38.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-09-16T08:47:38.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-09-16T08:47:38.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-09-16T08:47:38.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-09-16T08:47:38.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-09-16T08:47:38.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (9 ms) 2024-09-16T08:47:38.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-09-16T08:47:38.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (4 ms) 2024-09-16T08:47:38.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1154 ms total) 2024-09-16T08:47:38.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: 2024-09-16T08:47:38.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [----------] Global test environment tear-down 2024-09-16T08:47:38.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [==========] 24 tests from 2 test suites ran. (9693 ms total) 2024-09-16T08:47:38.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ PASSED ] 24 tests. 2024-09-16T08:47:38.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: Running main() from gmock_main.cc 2024-09-16T08:47:38.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-09-16T08:47:38.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [----------] Global test environment set-up. 2024-09-16T08:47:38.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-09-16T08:47:38.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-09-16T08:47:38.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStat.Stat (397 ms) 2024-09-16T08:47:38.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-09-16T08:47:38.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (11 ms) 2024-09-16T08:47:38.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-09-16T08:47:38.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStat.StatNS (17 ms) 2024-09-16T08:47:38.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-09-16T08:47:38.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (0 ms) 2024-09-16T08:47:38.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-09-16T08:47:38.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (7 ms) 2024-09-16T08:47:38.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [----------] 5 tests from LibRadosStat (432 ms total) 2024-09-16T08:47:38.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: 2024-09-16T08:47:38.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-09-16T08:47:38.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-09-16T08:47:38.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1105 ms) 2024-09-16T08:47:38.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-09-16T08:47:38.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (21 ms) 2024-09-16T08:47:38.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-09-16T08:47:38.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (0 ms) 2024-09-16T08:47:38.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-09-16T08:47:38.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (2 ms) 2024-09-16T08:47:38.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1128 ms total) 2024-09-16T08:47:38.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: 2024-09-16T08:47:38.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [----------] Global test environment tear-down 2024-09-16T08:47:38.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (9540 ms total) 2024-09-16T08:47:38.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ PASSED ] 9 tests. 2024-09-16T08:47:38.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-09-16T08:47:38.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-09-16T08:47:38.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-09-16T08:47:38.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-09-16T08:47:38.226 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: seed 76142 2024-09-16T08:47:38.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-09-16T08:47:38.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (282 ms) 2024-09-16T08:47:38.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-09-16T08:47:38.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (40 ms) 2024-09-16T08:47:38.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-09-16T08:47:38.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-09-16T08:47:38.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-09-16T08:47:38.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (98 ms) 2024-09-16T08:47:38.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-09-16T08:47:38.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (12 ms) 2024-09-16T08:47:38.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (432 ms total) 2024-09-16T08:47:38.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: 2024-09-16T08:47:38.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-09-16T08:47:38.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-09-16T08:47:38.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1102 ms) 2024-09-16T08:47:38.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-09-16T08:47:38.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (1 ms) 2024-09-16T08:47:38.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-09-16T08:47:38.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (3 ms) 2024-09-16T08:47:38.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-09-16T08:47:38.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (15 ms) 2024-09-16T08:47:38.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1121 ms total) 2024-09-16T08:47:38.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: 2024-09-16T08:47:38.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-09-16T08:47:38.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (9563 ms total) 2024-09-16T08:47:38.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-09-16T08:47:38.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: pgmap v60: 924 pgs: 32 creating+peering, 440 unknown, 452 active+clean; 145 MiB data, 715 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-09-16T08:47:38.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.25099 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-16T08:47:38.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:38.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:47:38.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:47:38.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.25381 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi160-76956-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi160-76956-2"}]': finished 2024-09-16T08:47:38.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-4-cache", "mode": "writeback"}]': finished 2024-09-16T08:47:38.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.35102 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"}]': finished 2024-09-16T08:47:38.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:38.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.35093 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"}]': finished 2024-09-16T08:47:38.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.35090 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"}]': finished 2024-09-16T08:47:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: osdmap e75: 8 total, 8 up, 8 in 2024-09-16T08:47:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76317-6", "pg_num": 4} : dispatch 2024-09-16T08:47:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76317-6", "pg_num": 4} : dispatch 2024-09-16T08:47:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-4"} : dispatch 2024-09-16T08:47:38.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[30137]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-4"} : dispatch 2024-09-16T08:47:38.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: pgmap v60: 924 pgs: 32 creating+peering, 440 unknown, 452 active+clean; 145 MiB data, 715 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-09-16T08:47:38.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.25099 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-16T08:47:38.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:38.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:47:38.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:47:38.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.25381 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi160-76956-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi160-76956-2"}]': finished 2024-09-16T08:47:38.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-4-cache", "mode": "writeback"}]': finished 2024-09-16T08:47:38.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.35102 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"}]': finished 2024-09-16T08:47:38.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:38.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.35093 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"}]': finished 2024-09-16T08:47:38.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.35090 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"}]': finished 2024-09-16T08:47:38.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:38.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:38.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:38.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: osdmap e75: 8 total, 8 up, 8 in 2024-09-16T08:47:38.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:38.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:38.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:38.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:38.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:38.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76317-6", "pg_num": 4} : dispatch 2024-09-16T08:47:38.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76317-6", "pg_num": 4} : dispatch 2024-09-16T08:47:38.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-4"} : dispatch 2024-09-16T08:47:38.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:38 smithi160 ceph-mon[40028]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-4"} : dispatch 2024-09-16T08:47:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: pgmap v60: 924 pgs: 32 creating+peering, 440 unknown, 452 active+clean; 145 MiB data, 715 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-09-16T08:47:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.25099 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-16T08:47:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:47:38.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:47:38.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.25381 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi160-76956-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi160-76956-2"}]': finished 2024-09-16T08:47:38.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-4-cache", "mode": "writeback"}]': finished 2024-09-16T08:47:38.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.35102 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-75783-16"}]': finished 2024-09-16T08:47:38.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:38.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.35093 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-76142-7"}]': finished 2024-09-16T08:47:38.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.35090 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-76254-7"}]': finished 2024-09-16T08:47:38.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2507420888' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:38.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/876778994' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:38.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/734453802' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:38.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: osdmap e75: 8 total, 8 up, 8 in 2024-09-16T08:47:38.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:38.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.35093 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"} : dispatch 2024-09-16T08:47:38.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.35102 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"} : dispatch 2024-09-16T08:47:38.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.35090 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"} : dispatch 2024-09-16T08:47:38.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:38.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76317-6", "pg_num": 4} : dispatch 2024-09-16T08:47:38.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76317-6", "pg_num": 4} : dispatch 2024-09-16T08:47:38.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-4"} : dispatch 2024-09-16T08:47:38.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:38 smithi169 ceph-mon[33465]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-4"} : dispatch 2024-09-16T08:47:39.170 INFO:tasks.workunit.client.0.smithi160.stdout:12532+0000 mon.b [INF] from='client.? v1:172.21.15.160:0/767158736' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.171 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.212578+0000 mon.a [INF] from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-4", "overlaypool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:39.171 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.212896+0000 mon.a [INF] from='client.25129 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi160-75757-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-16T08:47:39.171 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.213135+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/4027487533' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.171 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.214403+0000 mon.a [INF] from='client.25387 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.171 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.215170+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/1832056001' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi160-75741-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.171 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.215958+0000 mon.a [INF] from='client.35303 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.171 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.216248+0000 mon.a [INF] from='client.35330 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi160-75741-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.172 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.261197+0000 mon.c [INF] from='client.? v1:172.21.15.160:0/841615988' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi160-77061-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:39.172 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.269721+0000 mon.a [INF] from='client.35348 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi160-77061-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:39.172 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.373153+0000 mon.b [INF] from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi160-75791-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-16T08:47:39.172 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-16T08:47:35.373423+0000 mon.a [INF] from='client.25111 ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi160-75791-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-16T08:47:39.172 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (7998 ms) 2024-09-16T08:47:39.172 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (10468 ms total) 2024-09-16T08:47:39.172 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: 2024-09-16T08:47:39.172 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [----------] Global test environment tear-down 2024-09-16T08:47:39.173 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (10468 ms total) 2024-09-16T08:47:39.173 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ PASSED ] 4 tests. 2024-09-16T08:47:39.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:39.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.35348 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi160-77061-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi160-77061-2"}]': finished 2024-09-16T08:47:39.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.35093 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"}]': finished 2024-09-16T08:47:39.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.35102 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"}]': finished 2024-09-16T08:47:39.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.35090 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"}]': finished 2024-09-16T08:47:39.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:39.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76317-6", "pg_num": 4}]': finished 2024-09-16T08:47:39.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-4"}]': finished 2024-09-16T08:47:39.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:47:39.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:39.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:39.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: osdmap e76: 8 total, 8 up, 8 in 2024-09-16T08:47:39.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1296202690' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:39.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:47:39.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/474650742' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi160-75741-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:39.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2633236601' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:39.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.35348 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi160-77061-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi160-77061-2"}]': finished 2024-09-16T08:47:39.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.35093 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"}]': finished 2024-09-16T08:47:39.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.35102 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"}]': finished 2024-09-16T08:47:39.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.35090 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"}]': finished 2024-09-16T08:47:39.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:39.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76317-6", "pg_num": 4}]': finished 2024-09-16T08:47:39.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-4"}]': finished 2024-09-16T08:47:39.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:47:39.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:39.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:39.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: osdmap e76: 8 total, 8 up, 8 in 2024-09-16T08:47:39.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1296202690' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:39.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:47:39.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/474650742' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi160-75741-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:39.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2633236601' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.25450 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.25444 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi160-75741-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.25456 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:47:39.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:39.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:47:39.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"}]': finished 2024-09-16T08:47:39.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.25450 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:39.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.25444 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi160-75741-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:39.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.25456 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:39.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: osdmap e77: 8 total, 8 up, 8 in 2024-09-16T08:47:39.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "overlaypool": "test-rados-api-smithi160-76317-6"} : dispatch 2024-09-16T08:47:39.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "overlaypool": "test-rados-api-smithi160-76317-6"} : dispatch 2024-09-16T08:47:39.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1131943809' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:39.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[40028]: from='client.25381 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:39.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.25450 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.25444 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi160-75741-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.25456 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:47:39.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:39.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:47:39.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"}]': finished 2024-09-16T08:47:39.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.25450 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:39.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.25444 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi160-75741-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:39.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.25456 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:39.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: osdmap e77: 8 total, 8 up, 8 in 2024-09-16T08:47:39.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "overlaypool": "test-rados-api-smithi160-76317-6"} : dispatch 2024-09-16T08:47:39.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "overlaypool": "test-rados-api-smithi160-76317-6"} : dispatch 2024-09-16T08:47:39.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1131943809' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:39.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:39 smithi160 ceph-mon[30137]: from='client.25381 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:39.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:39.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.35348 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi160-77061-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi160-77061-2"}]': finished 2024-09-16T08:47:39.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.35093 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-76142-7"}]': finished 2024-09-16T08:47:39.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.35102 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-75783-16"}]': finished 2024-09-16T08:47:39.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.35090 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-76254-7"}]': finished 2024-09-16T08:47:39.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:39.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76317-6", "pg_num": 4}]': finished 2024-09-16T08:47:39.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-4"}]': finished 2024-09-16T08:47:39.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:47:39.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3580982754' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:39.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2450401756' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:39.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: osdmap e76: 8 total, 8 up, 8 in 2024-09-16T08:47:39.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1296202690' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.25111 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:39.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:47:39.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/474650742' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi160-75741-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.34952 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"} : dispatch 2024-09-16T08:47:39.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2633236601' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.25450 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.25444 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi160-75741-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.25456 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:39.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:47:39.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:39.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:47:39.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.34952 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-4", "tierpool": "test-rados-api-smithi160-76079-4-cache"}]': finished 2024-09-16T08:47:39.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.25450 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:39.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.25444 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi160-75741-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:39.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.25456 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:39.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: osdmap e77: 8 total, 8 up, 8 in 2024-09-16T08:47:39.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "overlaypool": "test-rados-api-smithi160-76317-6"} : dispatch 2024-09-16T08:47:39.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "overlaypool": "test-rados-api-smithi160-76317-6"} : dispatch 2024-09-16T08:47:39.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1131943809' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:39.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:39 smithi169 ceph-mon[33465]: from='client.25381 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:40.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: pgmap v63: 776 pgs: 204 creating+peering, 572 active+clean; 145 MiB data, 728 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-09-16T08:47:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "overlaypool": "test-rados-api-smithi160-76317-6"}]': finished 2024-09-16T08:47:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.25381 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi160-76956-2"}]': finished 2024-09-16T08:47:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.25474 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1131943809' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: osdmap e78: 8 total, 8 up, 8 in 2024-09-16T08:47:40.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76317-6", "mode": "writeback"} : dispatch 2024-09-16T08:47:40.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:40.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/865594382' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi160-76059-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:40.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:40.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: pgmap v63: 776 pgs: 204 creating+peering, 572 active+clean; 145 MiB data, 728 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-09-16T08:47:40.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:40.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:40.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:40.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "overlaypool": "test-rados-api-smithi160-76317-6"}]': finished 2024-09-16T08:47:40.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.25381 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi160-76956-2"}]': finished 2024-09-16T08:47:40.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.25474 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:40.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1131943809' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:40.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: osdmap e78: 8 total, 8 up, 8 in 2024-09-16T08:47:40.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76317-6", "mode": "writeback"} : dispatch 2024-09-16T08:47:40.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:40.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/865594382' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi160-76059-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:40.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:40.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76317-6", "mode": "writeback"} : dispatch 2024-09-16T08:47:40.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:40.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/841615988' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:40.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:40.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.25381 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:40.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.35423 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi160-76059-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:40.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.35429 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:40.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:40.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.35348 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:40.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:40.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76317-6", "mode": "writeback"} : dispatch 2024-09-16T08:47:40.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:40.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/841615988' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:40.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:40.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.25381 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:40.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.35423 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi160-76059-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:40.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.35429 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:40.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:40.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.35348 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:40.453 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: pgmap v63: 776 pgs: 204 creating+peering, 572 active+clean; 145 MiB data, 728 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-09-16T08:47:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "overlaypool": "test-rados-api-smithi160-76317-6"}]': finished 2024-09-16T08:47:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.25381 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi160-76956-2"}]': finished 2024-09-16T08:47:40.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.25474 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:40.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1131943809' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:40.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: osdmap e78: 8 total, 8 up, 8 in 2024-09-16T08:47:40.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76317-6", "mode": "writeback"} : dispatch 2024-09-16T08:47:40.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:40.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/865594382' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi160-76059-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:40.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:40.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76317-6", "mode": "writeback"} : dispatch 2024-09-16T08:47:40.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:40.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/841615988' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:40.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:40.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.25381 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi160-76956-2"} : dispatch 2024-09-16T08:47:40.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.35423 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi160-76059-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:40.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:40.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.35429 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:40.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:40.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.35348 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:40.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:41.108 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: Running main() from gmock_main.cc 2024-09-16T08:47:41.108 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-09-16T08:47:41.108 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [----------] Global test environment set-up. 2024-09-16T08:47:41.109 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-09-16T08:47:41.109 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-09-16T08:47:41.109 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (5260 ms) 2024-09-16T08:47:41.109 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-09-16T08:47:41.109 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (6925 ms) 2024-09-16T08:47:41.109 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (12185 ms total) 2024-09-16T08:47:41.109 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: 2024-09-16T08:47:41.109 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [----------] Global test environment tear-down 2024-09-16T08:47:41.109 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (12185 ms total) 2024-09-16T08:47:41.109 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [ PASSED ] 2 tests. 2024-09-16T08:47:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:47:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76317-6", "mode": "writeback"}]': finished 2024-09-16T08:47:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.35150 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10"}]': finished 2024-09-16T08:47:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.25381 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi160-76956-2"}]': finished 2024-09-16T08:47:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.35423 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi160-76059-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.35429 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.25204 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi160-75827-10"}]': finished 2024-09-16T08:47:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.35348 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi160-77061-2"}]': finished 2024-09-16T08:47:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/841615988' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: osdmap e79: 8 total, 8 up, 8 in 2024-09-16T08:47:41.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:41.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/161193294' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi160-75741-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/277218813' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:41.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.35348 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:41.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1137571276' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:41.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.35450 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi160-75741-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.35441 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:41.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix":"osd dump"} : dispatch 2024-09-16T08:47:41.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:41 smithi169 ceph-mon[33465]: from='client.35429 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:47:41.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76317-6", "mode": "writeback"}]': finished 2024-09-16T08:47:41.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.35150 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10"}]': finished 2024-09-16T08:47:41.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.25381 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi160-76956-2"}]': finished 2024-09-16T08:47:41.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.35423 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi160-76059-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:41.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.35429 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:41.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.25204 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi160-75827-10"}]': finished 2024-09-16T08:47:41.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.35348 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi160-77061-2"}]': finished 2024-09-16T08:47:41.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:41.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/841615988' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:41.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: osdmap e79: 8 total, 8 up, 8 in 2024-09-16T08:47:41.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:41.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/161193294' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi160-75741-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/277218813' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:41.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.35348 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:41.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1137571276' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:41.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.35450 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi160-75741-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.35441 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:47:41.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76317-6", "mode": "writeback"}]': finished 2024-09-16T08:47:41.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.35150 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi160-75849-10"}]': finished 2024-09-16T08:47:41.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.25381 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi160-76956-2"}]': finished 2024-09-16T08:47:41.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.35423 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi160-76059-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:41.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.35429 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:41.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.25204 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi160-75827-10"}]': finished 2024-09-16T08:47:41.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.35348 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi160-77061-2"}]': finished 2024-09-16T08:47:41.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4185753458' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:41.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/841615988' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:41.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: osdmap e79: 8 total, 8 up, 8 in 2024-09-16T08:47:41.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2911540179' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:41.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/161193294' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi160-75741-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/277218813' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.25204 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi160-75827-10"} : dispatch 2024-09-16T08:47:41.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.35348 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi160-77061-2"} : dispatch 2024-09-16T08:47:41.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1137571276' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.35150 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi160-75849-10"} : dispatch 2024-09-16T08:47:41.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.35450 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi160-75741-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.35441 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:41.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix":"osd dump"} : dispatch 2024-09-16T08:47:41.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[30137]: from='client.35429 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:41.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix":"osd dump"} : dispatch 2024-09-16T08:47:41.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:41.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:41 smithi160 ceph-mon[40028]: from='client.35429 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:42.115 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: Running main() from gmock_main.cc 2024-09-16T08:47:42.115 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-09-16T08:47:42.115 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [----------] Global test environment set-up. 2024-09-16T08:47:42.115 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-09-16T08:47:42.115 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-09-16T08:47:42.115 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (260 ms) 2024-09-16T08:47:42.116 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-09-16T08:47:42.116 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLock.LockShared (116 ms) 2024-09-16T08:47:42.116 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-09-16T08:47:42.116 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1017 ms) 2024-09-16T08:47:42.116 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-09-16T08:47:42.116 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1028 ms) 2024-09-16T08:47:42.116 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-09-16T08:47:42.116 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (21 ms) 2024-09-16T08:47:42.116 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-09-16T08:47:42.116 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLock.Unlock (7 ms) 2024-09-16T08:47:42.116 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-09-16T08:47:42.116 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (12 ms) 2024-09-16T08:47:42.117 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-09-16T08:47:42.117 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (2 ms) 2024-09-16T08:47:42.117 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [----------] 8 tests from LibRadosLock (2463 ms total) 2024-09-16T08:47:42.117 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: 2024-09-16T08:47:42.117 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-09-16T08:47:42.117 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-09-16T08:47:42.117 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1154 ms) 2024-09-16T08:47:42.117 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-09-16T08:47:42.117 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (4 ms) 2024-09-16T08:47:42.118 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-09-16T08:47:42.118 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1024 ms) 2024-09-16T08:47:42.118 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-09-16T08:47:42.118 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1003 ms) 2024-09-16T08:47:42.118 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-09-16T08:47:42.118 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (4 ms) 2024-09-16T08:47:42.118 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-09-16T08:47:42.118 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (3 ms) 2024-09-16T08:47:42.118 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-09-16T08:47:42.118 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (5 ms) 2024-09-16T08:47:42.118 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-09-16T08:47:42.118 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (2 ms) 2024-09-16T08:47:42.118 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3199 ms total) 2024-09-16T08:47:42.118 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: 2024-09-16T08:47:42.118 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [----------] Global test environment tear-down 2024-09-16T08:47:42.119 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (13570 ms total) 2024-09-16T08:47:42.119 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ PASSED ] 16 tests. 2024-09-16T08:47:42.119 INFO:tasks.workunit.client.0.smithi160.stdout: pool: Running main() from gmock_main.cc 2024-09-16T08:47:42.119 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-09-16T08:47:42.119 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [----------] Global test environment set-up. 2024-09-16T08:47:42.119 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-09-16T08:47:42.119 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-09-16T08:47:42.119 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ OK ] NeoRadosPools.PoolList (1225 ms) 2024-09-16T08:47:42.119 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-09-16T08:47:42.119 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (2002 ms) 2024-09-16T08:47:42.119 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-09-16T08:47:42.119 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (2035 ms) 2024-09-16T08:47:42.119 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-09-16T08:47:42.120 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (4003 ms) 2024-09-16T08:47:42.120 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-09-16T08:47:42.120 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (1901 ms) 2024-09-16T08:47:42.120 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-09-16T08:47:42.120 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (2000 ms) 2024-09-16T08:47:42.120 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [----------] 6 tests from NeoRadosPools (13166 ms total) 2024-09-16T08:47:42.120 INFO:tasks.workunit.client.0.smithi160.stdout: pool: 2024-09-16T08:47:42.120 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [----------] Global test environment tear-down 2024-09-16T08:47:42.120 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [==========] 6 tests from 1 test suite ran. (13166 ms total) 2024-09-16T08:47:42.120 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ PASSED ] 6 tests. 2024-09-16T08:47:42.124 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-09-16T08:47:42.124 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-09-16T08:47:42.124 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-09-16T08:47:42.124 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-09-16T08:47:42.124 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: seed 75849 2024-09-16T08:47:42.124 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-09-16T08:47:42.124 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (351 ms) 2024-09-16T08:47:42.125 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-09-16T08:47:42.125 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (19 ms) 2024-09-16T08:47:42.125 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-09-16T08:47:42.125 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1036 ms) 2024-09-16T08:47:42.125 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-09-16T08:47:42.125 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1028 ms) 2024-09-16T08:47:42.125 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-09-16T08:47:42.125 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (10 ms) 2024-09-16T08:47:42.125 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-09-16T08:47:42.125 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (5 ms) 2024-09-16T08:47:42.125 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-09-16T08:47:42.125 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (4 ms) 2024-09-16T08:47:42.126 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-09-16T08:47:42.126 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (3 ms) 2024-09-16T08:47:42.126 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2456 ms total) 2024-09-16T08:47:42.126 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: 2024-09-16T08:47:42.126 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-09-16T08:47:42.126 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-09-16T08:47:42.126 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1083 ms) 2024-09-16T08:47:42.126 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-09-16T08:47:42.126 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (58 ms) 2024-09-16T08:47:42.126 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-09-16T08:47:42.126 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1025 ms) 2024-09-16T08:47:42.126 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-09-16T08:47:42.126 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1005 ms) 2024-09-16T08:47:42.126 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-09-16T08:47:42.126 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (4 ms) 2024-09-16T08:47:42.126 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-09-16T08:47:42.127 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (4 ms) 2024-09-16T08:47:42.127 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-09-16T08:47:42.127 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (5 ms) 2024-09-16T08:47:42.127 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-09-16T08:47:42.127 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (2 ms) 2024-09-16T08:47:42.127 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3186 ms total) 2024-09-16T08:47:42.127 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: 2024-09-16T08:47:42.127 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-09-16T08:47:42.127 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (13563 ms total) 2024-09-16T08:47:42.127 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-09-16T08:47:42.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: pgmap v66: 584 pgs: 128 unknown, 4 creating+peering, 452 active+clean; 145 MiB data, 728 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-09-16T08:47:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.25474 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.25204 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi160-75827-10"}]': finished 2024-09-16T08:47:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.35348 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi160-77061-2"}]': finished 2024-09-16T08:47:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1137571276' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.35150 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi160-75849-10"}]': finished 2024-09-16T08:47:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.35450 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi160-75741-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.35441 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.35429 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: osdmap e80: 8 total, 8 up, 8 in 2024-09-16T08:47:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app2"} : dispatch 2024-09-16T08:47:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:42.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.35429 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:42.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1330230264' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi160-77061-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:42.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.25531 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi160-77061-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:42.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:42.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:42.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:42.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:42.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:42.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: pgmap v66: 584 pgs: 128 unknown, 4 creating+peering, 452 active+clean; 145 MiB data, 728 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-09-16T08:47:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.25474 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.25204 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi160-75827-10"}]': finished 2024-09-16T08:47:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.35348 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi160-77061-2"}]': finished 2024-09-16T08:47:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1137571276' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.35150 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi160-75849-10"}]': finished 2024-09-16T08:47:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.35450 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi160-75741-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.35441 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.35429 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:42.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: osdmap e80: 8 total, 8 up, 8 in 2024-09-16T08:47:42.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app2"} : dispatch 2024-09-16T08:47:42.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:42.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.35429 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:42.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1330230264' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi160-77061-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:42.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.25531 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi160-77061-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:42.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:42.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:42.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:42.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: pgmap v66: 584 pgs: 128 unknown, 4 creating+peering, 452 active+clean; 145 MiB data, 728 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-09-16T08:47:42.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.25474 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-75791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:42.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.25204 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi160-75827-10"}]': finished 2024-09-16T08:47:42.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.35348 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi160-77061-2"}]': finished 2024-09-16T08:47:42.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1137571276' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:42.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.35150 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi160-75849-10"}]': finished 2024-09-16T08:47:42.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.35450 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi160-75741-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:42.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.35441 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:42.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.35429 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:42.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: osdmap e80: 8 total, 8 up, 8 in 2024-09-16T08:47:42.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app2"} : dispatch 2024-09-16T08:47:42.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:42.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.35429 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:42.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1330230264' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi160-77061-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:42.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.25531 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi160-77061-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:42.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:42.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:42.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:42.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:42.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:42.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:42.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:42.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:42.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:43.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:43 smithi169 ceph-mon[33465]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:43 smithi169 ceph-mon[33465]: from='client.35429 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:43 smithi169 ceph-mon[33465]: from='client.25531 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi160-77061-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:43 smithi169 ceph-mon[33465]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1330230264' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi160-77061-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:43 smithi169 ceph-mon[33465]: osdmap e81: 8 total, 8 up, 8 in 2024-09-16T08:47:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi160-76301-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:43 smithi169 ceph-mon[33465]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi160-76301-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:43 smithi169 ceph-mon[33465]: from='client.25531 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi160-77061-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:43.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"dne","key":"key","value":"value"} : dispatch 2024-09-16T08:47:43.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-16T08:47:43.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:43 smithi169 ceph-mon[33465]: from='client.35429 ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-16T08:47:43.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:43.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[30137]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[30137]: from='client.35429 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[30137]: from='client.25531 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi160-77061-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[30137]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1330230264' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi160-77061-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[30137]: osdmap e81: 8 total, 8 up, 8 in 2024-09-16T08:47:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi160-76301-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[30137]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi160-76301-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[30137]: from='client.25531 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi160-77061-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"dne","key":"key","value":"value"} : dispatch 2024-09-16T08:47:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-16T08:47:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[30137]: from='client.35429 ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-16T08:47:43.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[40028]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[40028]: from='client.35429 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi160-75919-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[40028]: from='client.25531 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi160-77061-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[40028]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1330230264' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi160-77061-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[40028]: osdmap e81: 8 total, 8 up, 8 in 2024-09-16T08:47:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi160-76301-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[40028]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi160-76301-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[40028]: from='client.25531 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi160-77061-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:43.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"dne","key":"key","value":"value"} : dispatch 2024-09-16T08:47:43.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-16T08:47:43.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[40028]: from='client.35429 ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-16T08:47:43.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:44.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-09-16T08:47:44.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-09-16T08:47:44.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-09-16T08:47:44.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-09-16T08:47:44.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-09-16T08:47:44.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: notify 2024-09-16T08:47:44.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1095 ms) 2024-09-16T08:47:44.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-09-16T08:47:44.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (9 ms) 2024-09-16T08:47:44.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (1104 ms total) 2024-09-16T08:47:44.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: 2024-09-16T08:47:44.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-09-16T08:47:44.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-09-16T08:47:44.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: notify 2024-09-16T08:47:44.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (138 ms) 2024-09-16T08:47:44.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-09-16T08:47:44.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: notify 2024-09-16T08:47:44.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (3852 ms) 2024-09-16T08:47:44.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-09-16T08:47:44.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (5 ms) 2024-09-16T08:47:44.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-09-16T08:47:44.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (4 ms) 2024-09-16T08:47:44.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-09-16T08:47:44.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 94060058777472 notify_id 339302416387 notifier_gid 35345 2024-09-16T08:47:44.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (6 ms) 2024-09-16T08:47:44.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-09-16T08:47:44.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 94060058797872 notify_id 339302416385 notifier_gid 35345 2024-09-16T08:47:44.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (4 ms) 2024-09-16T08:47:44.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-09-16T08:47:44.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 94060058797872 notify_id 339302416388 notifier_gid 35345 2024-09-16T08:47:44.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (5 ms) 2024-09-16T08:47:44.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-09-16T08:47:44.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 94060058797872 notify_id 339302416389 notifier_gid 35345 2024-09-16T08:47:44.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (4 ms) 2024-09-16T08:47:44.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-09-16T08:47:44.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 94060058803616 notify_id 339302416390 notifier_gid 35345 2024-09-16T08:47:44.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (7 ms) 2024-09-16T08:47:44.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-09-16T08:47:44.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 94060058797872 notify_id 339302416385 notifier_gid 35345 2024-09-16T08:47:44.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (13 ms) 2024-09-16T08:47:44.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-09-16T08:47:44.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: trying... 2024-09-16T08:47:44.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 94060058797872 notify_id 339302416386 notifier_gid 35345 2024-09-16T08:47:44.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: timed out 2024-09-16T08:47:44.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: flushing 2024-09-16T08:47:44.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: pgmap v69: 656 pgs: 296 unknown, 4 creating+peering, 356 active+clean; 145 MiB data, 728 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:47:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:47:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: pool 'PoolQuotaPP_smithi160-75757-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-16T08:47:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-16T08:47:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: from='client.35429 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-16T08:47:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-16T08:47:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: osdmap e82: 8 total, 8 up, 8 in 2024-09-16T08:47:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3071666208' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi160-75741-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3243473080' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3233884252' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi160-76133-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3585035781' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: from='client.35429 ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-16T08:47:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: from='client.35507 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: from='client.25570 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi160-75741-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: from='client.25582 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi160-76133-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: from='client.25558 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:44.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: pgmap v69: 656 pgs: 296 unknown, 4 creating+peering, 356 active+clean; 145 MiB data, 728 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:47:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:47:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: pool 'PoolQuotaPP_smithi160-75757-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-16T08:47:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-16T08:47:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: from='client.35429 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-16T08:47:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-16T08:47:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: osdmap e82: 8 total, 8 up, 8 in 2024-09-16T08:47:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3071666208' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi160-75741-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3243473080' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3233884252' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi160-76133-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3585035781' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: from='client.35429 ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-16T08:47:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: from='client.35507 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: from='client.25570 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi160-75741-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: from='client.25582 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi160-76133-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: from='client.25558 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: pgmap v69: 656 pgs: 296 unknown, 4 creating+peering, 356 active+clean; 145 MiB data, 728 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:47:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:47:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: pool 'PoolQuotaPP_smithi160-75757-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-16T08:47:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-16T08:47:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: from='client.35429 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-16T08:47:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-16T08:47:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: osdmap e82: 8 total, 8 up, 8 in 2024-09-16T08:47:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3071666208' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi160-75741-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3243473080' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3233884252' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi160-76133-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3585035781' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: from='client.35429 ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-16T08:47:44.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:44.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: from='client.35507 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: from='client.25570 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi160-75741-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: from='client.25582 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi160-76133-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: from='client.25558 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:44.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:45.099 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:47:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:47:45.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:47:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:47:46.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_wat_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-09-16T08:47:46.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (2 ms) 2024-09-16T08:47:46.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-09-16T08:47:46.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (4 ms) 2024-09-16T08:47:46.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-09-16T08:47:46.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (3 ms) 2024-09-16T08:47:46.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-09-16T08:47:46.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (6 ms) 2024-09-16T08:47:46.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-09-16T08:47:46.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (5 ms) 2024-09-16T08:47:46.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-09-16T08:47:46.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (5 ms) 2024-09-16T08:47:46.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-09-16T08:47:46.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (6 ms) 2024-09-16T08:47:46.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-09-16T08:47:46.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (2 ms) 2024-09-16T08:47:46.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-09-16T08:47:46.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (3 ms) 2024-09-16T08:47:46.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-09-16T08:47:46.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (7 ms) 2024-09-16T08:47:46.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CrcZeroWrite 2024-09-16T08:47:46.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CrcZeroWrite (6737 ms) 2024-09-16T08:47:46.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-09-16T08:47:46.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (1092 ms) 2024-09-16T08:47:46.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-09-16T08:47:46.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (34 ms) 2024-09-16T08:47:46.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-09-16T08:47:46.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (7 ms) 2024-09-16T08:47:46.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-09-16T08:47:46.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (11 ms) 2024-09-16T08:47:46.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [----------] 18 tests from LibRadosIoECPP (9074 ms total) 2024-09-16T08:47:46.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: 2024-09-16T08:47:46.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [----------] Global test environment tear-down 2024-09-16T08:47:46.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [==========] 40 tests from 2 test suites ran. (17602 ms total) 2024-09-16T08:47:46.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ PASSED ] 40 tests. 2024-09-16T08:47:46.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:46.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: pgmap v72: 680 pgs: 64 creating+peering, 224 unknown, 392 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-16T08:47:46.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi160-76301-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"}]': finished 2024-09-16T08:47:46.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: from='client.25531 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi160-77061-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi160-77061-3"}]': finished 2024-09-16T08:47:46.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: from='client.35429 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-16T08:47:46.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: from='client.25474 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:46.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: from='client.35507 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:46.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: from='client.25570 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi160-75741-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:46.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: from='client.25582 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi160-76133-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:46.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: from='client.25558 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:46.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: osdmap e83: 8 total, 8 up, 8 in 2024-09-16T08:47:46.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:46.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1"} : dispatch 2024-09-16T08:47:46.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:46.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:46.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: from='client.35429 ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1"} : dispatch 2024-09-16T08:47:46.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:46.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:46.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: pgmap v72: 680 pgs: 64 creating+peering, 224 unknown, 392 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-16T08:47:46.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi160-76301-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"}]': finished 2024-09-16T08:47:46.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: from='client.25531 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi160-77061-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi160-77061-3"}]': finished 2024-09-16T08:47:46.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: from='client.35429 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-16T08:47:46.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: from='client.25474 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:46.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: from='client.35507 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:46.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: from='client.25570 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi160-75741-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:46.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: from='client.25582 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi160-76133-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:46.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: from='client.25558 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:46.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: osdmap e83: 8 total, 8 up, 8 in 2024-09-16T08:47:46.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:46.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1"} : dispatch 2024-09-16T08:47:46.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:46.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:46.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: from='client.35429 ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1"} : dispatch 2024-09-16T08:47:46.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:46.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:46.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: pgmap v72: 680 pgs: 64 creating+peering, 224 unknown, 392 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-16T08:47:46.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi160-76301-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"}]': finished 2024-09-16T08:47:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: from='client.25531 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi160-77061-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi160-77061-3"}]': finished 2024-09-16T08:47:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: from='client.35429 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-16T08:47:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: from='client.25474 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: from='client.35507 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: from='client.25570 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi160-75741-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: from='client.25582 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi160-76133-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: from='client.25558 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: osdmap e83: 8 total, 8 up, 8 in 2024-09-16T08:47:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/562925765' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2167235556' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1"} : dispatch 2024-09-16T08:47:46.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: from='client.25474 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"} : dispatch 2024-09-16T08:47:46.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:46.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: from='client.35429 ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1"} : dispatch 2024-09-16T08:47:46.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:47.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:47 smithi160 ceph-mon[30137]: from='client.25474 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:47.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:47.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:47 smithi160 ceph-mon[30137]: from='client.35429 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1"}]': finished 2024-09-16T08:47:47.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:47 smithi160 ceph-mon[30137]: osdmap e84: 8 total, 8 up, 8 in 2024-09-16T08:47:47.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:47.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:47:47.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:47 smithi160 ceph-mon[40028]: from='client.25474 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:47.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:47.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:47 smithi160 ceph-mon[40028]: from='client.35429 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1"}]': finished 2024-09-16T08:47:47.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:47 smithi160 ceph-mon[40028]: osdmap e84: 8 total, 8 up, 8 in 2024-09-16T08:47:47.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:47.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:47:47.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:47 smithi169 ceph-mon[33465]: from='client.25474 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-75791-24"}]': finished 2024-09-16T08:47:47.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:47.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:47 smithi169 ceph-mon[33465]: from='client.35429 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi160-75919-1","app":"app1","key":"key1"}]': finished 2024-09-16T08:47:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:47 smithi169 ceph-mon[33465]: osdmap e84: 8 total, 8 up, 8 in 2024-09-16T08:47:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:47:48.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[40028]: pgmap v75: 568 pgs: 64 creating+peering, 112 unknown, 392 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 7.7 KiB/s wr, 7 op/s 2024-09-16T08:47:48.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:47:48.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[40028]: osdmap e85: 8 total, 8 up, 8 in 2024-09-16T08:47:48.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3431739759' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:48.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-7"} : dispatch 2024-09-16T08:47:48.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[40028]: from='client.25600 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:48.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:48.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[40028]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:48.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1501808290' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi160-75741-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:48.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[40028]: from='client.35543 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi160-75741-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:48.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:48.453 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[30137]: pgmap v75: 568 pgs: 64 creating+peering, 112 unknown, 392 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 7.7 KiB/s wr, 7 op/s 2024-09-16T08:47:48.454 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:47:48.454 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[30137]: osdmap e85: 8 total, 8 up, 8 in 2024-09-16T08:47:48.454 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3431739759' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:48.454 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-7"} : dispatch 2024-09-16T08:47:48.454 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[30137]: from='client.25600 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:48.454 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:48.454 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[30137]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:48.454 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1501808290' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi160-75741-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:48.454 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[30137]: from='client.35543 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi160-75741-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:48.454 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:48.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:48 smithi169 ceph-mon[33465]: pgmap v75: 568 pgs: 64 creating+peering, 112 unknown, 392 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 7.7 KiB/s wr, 7 op/s 2024-09-16T08:47:48.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:47:48.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:48 smithi169 ceph-mon[33465]: osdmap e85: 8 total, 8 up, 8 in 2024-09-16T08:47:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3431739759' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-7"} : dispatch 2024-09-16T08:47:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:48 smithi169 ceph-mon[33465]: from='client.25600 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:48 smithi169 ceph-mon[33465]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1501808290' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi160-75741-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:48 smithi169 ceph-mon[33465]: from='client.35543 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi160-75741-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:49.194 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-09-16T08:47:49.194 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-09-16T08:47:49.194 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-09-16T08:47:49.194 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-09-16T08:47:49.194 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-09-16T08:47:49.194 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify_test_cb 2024-09-16T08:47:49.194 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (359 ms) 2024-09-16T08:47:49.194 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-09-16T08:47:49.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-16T08:47:49.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94355283717264 err -107 2024-09-16T08:47:49.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (1011 ms) 2024-09-16T08:47:49.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-09-16T08:47:49.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-16T08:47:49.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94355284451104 err -107 2024-09-16T08:47:49.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1017 ms) 2024-09-16T08:47:49.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-09-16T08:47:49.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_cb from 24772 notify_id 300647710720 cookie 94355284455344 2024-09-16T08:47:49.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (50 ms) 2024-09-16T08:47:49.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-09-16T08:47:49.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_cb from 24772 notify_id 300647710721 cookie 94355284480240 2024-09-16T08:47:49.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (13 ms) 2024-09-16T08:47:49.195 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-09-16T08:47:49.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_cb from 24772 notify_id 300647710721 cookie 94355284483424 2024-09-16T08:47:49.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (4 ms) 2024-09-16T08:47:49.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-09-16T08:47:49.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_cb from 24772 notify_id 300647710720 cookie 94355284483424 2024-09-16T08:47:49.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_cb from 24772 notify_id 300647710720 cookie 94355284502688 2024-09-16T08:47:49.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (7 ms) 2024-09-16T08:47:49.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-09-16T08:47:49.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_cb from 24772 notify_id 300647710722 cookie 94355284483424 2024-09-16T08:47:49.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_cb from 24772 notify_id 304942678019 cookie 94355284483424 2024-09-16T08:47:49.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3004 ms) 2024-09-16T08:47:49.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-09-16T08:47:49.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-09-16T08:47:49.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94355284483424 err -107 2024-09-16T08:47:49.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_cb from 24772 notify_id 335007449090 cookie 94355284483424 2024-09-16T08:47:49.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5008 ms) 2024-09-16T08:47:49.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-09-16T08:47:49.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-09-16T08:47:49.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94355284483424 err -107 2024-09-16T08:47:49.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1005 ms) 2024-09-16T08:47:49.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (11478 ms total) 2024-09-16T08:47:49.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: 2024-09-16T08:47:49.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-09-16T08:47:49.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-09-16T08:47:49.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify_test_cb 2024-09-16T08:47:49.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1153 ms) 2024-09-16T08:47:49.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1153 ms total) 2024-09-16T08:47:49.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: 2024-09-16T08:47:49.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-09-16T08:47:49.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (20496 ms total) 2024-09-16T08:47:49.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-09-16T08:47:49.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-7"}]': finished 2024-09-16T08:47:49.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='client.25600 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:49.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"}]': finished 2024-09-16T08:47:49.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='client.35543 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi160-75741-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:49.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: osdmap e86: 8 total, 8 up, 8 in 2024-09-16T08:47:49.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:49.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1330230264' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:49.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:49.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='client.25531 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:49.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:49.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:47:49.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:49.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:47:49.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:47:49.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi160-76301-12"}]': finished 2024-09-16T08:47:49.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='client.25531 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi160-77061-3"}]': finished 2024-09-16T08:47:49.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:47:49.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: osdmap e87: 8 total, 8 up, 8 in 2024-09-16T08:47:49.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-7"} : dispatch 2024-09-16T08:47:49.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[30137]: from='client.27361 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:49.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-7"}]': finished 2024-09-16T08:47:49.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='client.25600 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:49.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"}]': finished 2024-09-16T08:47:49.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='client.35543 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi160-75741-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:49.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: osdmap e86: 8 total, 8 up, 8 in 2024-09-16T08:47:49.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:49.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1330230264' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:49.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:49.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='client.25531 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:49.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:49.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:47:49.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:49.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:47:49.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:47:49.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi160-76301-12"}]': finished 2024-09-16T08:47:49.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='client.25531 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi160-77061-3"}]': finished 2024-09-16T08:47:49.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:47:49.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: osdmap e87: 8 total, 8 up, 8 in 2024-09-16T08:47:49.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-7"} : dispatch 2024-09-16T08:47:49.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:49 smithi160 ceph-mon[40028]: from='client.27361 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:49.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-7"}]': finished 2024-09-16T08:47:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='client.25600 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76511-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi160-76301-12"}]': finished 2024-09-16T08:47:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='client.35543 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi160-75741-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: osdmap e86: 8 total, 8 up, 8 in 2024-09-16T08:47:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/301025087' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1330230264' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='client.35495 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi160-76301-12"} : dispatch 2024-09-16T08:47:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='client.25531 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:47:49.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:47:49.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:47:49.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:47:49.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi160-76301-12"}]': finished 2024-09-16T08:47:49.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='client.25531 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi160-77061-3"}]': finished 2024-09-16T08:47:49.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:47:49.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: osdmap e87: 8 total, 8 up, 8 in 2024-09-16T08:47:49.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-7"} : dispatch 2024-09-16T08:47:49.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:49 smithi169 ceph-mon[33465]: from='client.27361 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:50.173 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: Running main() from gmock_main.cc 2024-09-16T08:47:50.173 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-09-16T08:47:50.173 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [----------] Global test environment set-up. 2024-09-16T08:47:50.173 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-09-16T08:47:50.173 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-09-16T08:47:50.173 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (6253 ms) 2024-09-16T08:47:50.173 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-09-16T08:47:50.173 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (6930 ms) 2024-09-16T08:47:50.173 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-09-16T08:47:50.174 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (8050 ms) 2024-09-16T08:47:50.174 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [----------] 3 tests from NeoradosECList (21233 ms total) 2024-09-16T08:47:50.174 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: 2024-09-16T08:47:50.174 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [----------] Global test environment tear-down 2024-09-16T08:47:50.174 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (21233 ms total) 2024-09-16T08:47:50.174 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [ PASSED ] 3 tests. 2024-09-16T08:47:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:50 smithi169 ceph-mon[33465]: pgmap v78: 712 pgs: 64 creating+peering, 648 active+clean; 145 MiB data, 806 MiB used, 714 GiB / 715 GiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-09-16T08:47:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/69010690' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1330230264' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:50 smithi169 ceph-mon[33465]: from='client.25531 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:50.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:50.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:50 smithi169 ceph-mon[33465]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-16T08:47:50.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-7"}]': finished 2024-09-16T08:47:50.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:50 smithi169 ceph-mon[33465]: from='client.27361 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:50.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:50 smithi169 ceph-mon[33465]: from='client.25531 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi160-77061-3"}]': finished 2024-09-16T08:47:50.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:50 smithi169 ceph-mon[33465]: osdmap e88: 8 total, 8 up, 8 in 2024-09-16T08:47:50.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3238740229' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi160-75741-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:50.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:50 smithi169 ceph-mon[33465]: from='client.28453 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi160-75741-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:50.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[30137]: pgmap v78: 712 pgs: 64 creating+peering, 648 active+clean; 145 MiB data, 806 MiB used, 714 GiB / 715 GiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-09-16T08:47:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/69010690' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1330230264' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[30137]: from='client.25531 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[30137]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-16T08:47:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-7"}]': finished 2024-09-16T08:47:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[30137]: from='client.27361 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[30137]: from='client.25531 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi160-77061-3"}]': finished 2024-09-16T08:47:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[30137]: osdmap e88: 8 total, 8 up, 8 in 2024-09-16T08:47:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3238740229' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi160-75741-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[30137]: from='client.28453 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi160-75741-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[40028]: pgmap v78: 712 pgs: 64 creating+peering, 648 active+clean; 145 MiB data, 806 MiB used, 714 GiB / 715 GiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-09-16T08:47:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/69010690' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1330230264' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[40028]: from='client.25531 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi160-77061-3"} : dispatch 2024-09-16T08:47:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[40028]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-16T08:47:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-7"}]': finished 2024-09-16T08:47:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[40028]: from='client.27361 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[40028]: from='client.25531 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi160-77061-3"}]': finished 2024-09-16T08:47:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[40028]: osdmap e88: 8 total, 8 up, 8 in 2024-09-16T08:47:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3238740229' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi160-75741-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:50.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:50 smithi160 ceph-mon[40028]: from='client.28453 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi160-75741-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:51.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:51.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:51 smithi169 ceph-mon[33465]: from='client.28453 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi160-75741-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:51.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:51 smithi169 ceph-mon[33465]: osdmap e89: 8 total, 8 up, 8 in 2024-09-16T08:47:51.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:51.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:51 smithi160 ceph-mon[30137]: from='client.28453 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi160-75741-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:51.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:51 smithi160 ceph-mon[30137]: osdmap e89: 8 total, 8 up, 8 in 2024-09-16T08:47:51.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:51.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:51 smithi160 ceph-mon[40028]: from='client.28453 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi160-75741-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:51.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:51 smithi160 ceph-mon[40028]: osdmap e89: 8 total, 8 up, 8 in 2024-09-16T08:47:52.183 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: Running main() from gmock_main.cc 2024-09-16T08:47:52.183 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-09-16T08:47:52.183 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [----------] Global test environment set-up. 2024-09-16T08:47:52.184 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-09-16T08:47:52.184 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-09-16T08:47:52.184 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (2198 ms) 2024-09-16T08:47:52.184 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-09-16T08:47:52.184 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (3026 ms) 2024-09-16T08:47:52.184 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-09-16T08:47:52.184 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (3022 ms) 2024-09-16T08:47:52.184 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-09-16T08:47:52.184 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (2888 ms) 2024-09-16T08:47:52.184 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-09-16T08:47:52.184 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (3014 ms) 2024-09-16T08:47:52.185 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-09-16T08:47:52.185 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (3017 ms) 2024-09-16T08:47:52.185 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-09-16T08:47:52.185 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (6031 ms) 2024-09-16T08:47:52.185 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (23196 ms total) 2024-09-16T08:47:52.185 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: 2024-09-16T08:47:52.185 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [----------] Global test environment tear-down 2024-09-16T08:47:52.185 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (23196 ms total) 2024-09-16T08:47:52.185 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ PASSED ] 7 tests. 2024-09-16T08:47:52.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-09-16T08:47:52.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-09-16T08:47:52.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-09-16T08:47:52.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-09-16T08:47:52.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-09-16T08:47:52.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-09-16T08:47:52.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-09-16T08:47:52.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: 2024-09-16T08:47:52.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-09-16T08:47:52.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-09-16T08:47:52.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (2417 ms) 2024-09-16T08:47:52.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-09-16T08:47:52.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (3002 ms) 2024-09-16T08:47:52.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-09-16T08:47:52.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (3035 ms) 2024-09-16T08:47:52.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-09-16T08:47:52.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (2896 ms) 2024-09-16T08:47:52.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-09-16T08:47:52.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3010 ms) 2024-09-16T08:47:52.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-09-16T08:47:52.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (2991 ms) 2024-09-16T08:47:52.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-09-16T08:47:52.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (6081 ms) 2024-09-16T08:47:52.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (23432 ms total) 2024-09-16T08:47:52.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: 2024-09-16T08:47:52.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-09-16T08:47:52.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (23434 ms total) 2024-09-16T08:47:52.204 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-09-16T08:47:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:52 smithi169 ceph-mon[33465]: pgmap v81: 680 pgs: 128 unknown, 32 creating+peering, 520 active+clean; 145 MiB data, 806 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T08:47:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:52 smithi169 ceph-mon[33465]: osdmap e90: 8 total, 8 up, 8 in 2024-09-16T08:47:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1791210908' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:52 smithi169 ceph-mon[33465]: from='client.30373 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:52 smithi160 ceph-mon[30137]: pgmap v81: 680 pgs: 128 unknown, 32 creating+peering, 520 active+clean; 145 MiB data, 806 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T08:47:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:52 smithi160 ceph-mon[30137]: osdmap e90: 8 total, 8 up, 8 in 2024-09-16T08:47:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1791210908' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:52 smithi160 ceph-mon[30137]: from='client.30373 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:52.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:52 smithi160 ceph-mon[40028]: pgmap v81: 680 pgs: 128 unknown, 32 creating+peering, 520 active+clean; 145 MiB data, 806 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T08:47:52.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:52.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:52 smithi160 ceph-mon[40028]: osdmap e90: 8 total, 8 up, 8 in 2024-09-16T08:47:52.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1791210908' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:52.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:52 smithi160 ceph-mon[40028]: from='client.30373 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:52.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:53.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:53.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:53 smithi169 ceph-mon[33465]: from='client.30373 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:53.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:53 smithi169 ceph-mon[33465]: osdmap e91: 8 total, 8 up, 8 in 2024-09-16T08:47:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/831395575' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi160-75741-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:53 smithi160 ceph-mon[30137]: from='client.30373 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:53 smithi160 ceph-mon[30137]: osdmap e91: 8 total, 8 up, 8 in 2024-09-16T08:47:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/831395575' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi160-75741-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:53 smithi160 ceph-mon[40028]: from='client.30373 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:53.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:53.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:53 smithi160 ceph-mon[40028]: osdmap e91: 8 total, 8 up, 8 in 2024-09-16T08:47:53.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/831395575' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi160-75741-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:54.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:54 smithi169 ceph-mon[33465]: pgmap v84: 552 pgs: 128 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 806 MiB used, 714 GiB / 715 GiB avail 2024-09-16T08:47:54.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:54.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4"} : dispatch 2024-09-16T08:47:54.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:47:54.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:54 smithi169 ceph-mon[33465]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4"} : dispatch 2024-09-16T08:47:54.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/831395575' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi160-75741-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:54.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:47:54.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:54 smithi169 ceph-mon[33465]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4"}]': finished 2024-09-16T08:47:54.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:54 smithi169 ceph-mon[33465]: osdmap e92: 8 total, 8 up, 8 in 2024-09-16T08:47:54.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6"} : dispatch 2024-09-16T08:47:54.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-9"} : dispatch 2024-09-16T08:47:54.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3145873442' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:54.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:54 smithi169 ceph-mon[33465]: from='client.32671 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:54.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:54 smithi169 ceph-mon[33465]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6"} : dispatch 2024-09-16T08:47:54.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[40028]: pgmap v84: 552 pgs: 128 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 806 MiB used, 714 GiB / 715 GiB avail 2024-09-16T08:47:54.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:54.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4"} : dispatch 2024-09-16T08:47:54.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:47:54.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[40028]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4"} : dispatch 2024-09-16T08:47:54.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/831395575' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi160-75741-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:54.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:47:54.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[40028]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4"}]': finished 2024-09-16T08:47:54.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[40028]: osdmap e92: 8 total, 8 up, 8 in 2024-09-16T08:47:54.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6"} : dispatch 2024-09-16T08:47:54.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-9"} : dispatch 2024-09-16T08:47:54.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3145873442' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:54.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[40028]: from='client.32671 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:54.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[40028]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6"} : dispatch 2024-09-16T08:47:54.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:54.705 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[30137]: pgmap v84: 552 pgs: 128 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 806 MiB used, 714 GiB / 715 GiB avail 2024-09-16T08:47:54.705 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:54.705 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4"} : dispatch 2024-09-16T08:47:54.705 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:47:54.705 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[30137]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4"} : dispatch 2024-09-16T08:47:54.705 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/831395575' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi160-75741-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:54.705 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:47:54.705 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[30137]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4"}]': finished 2024-09-16T08:47:54.705 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[30137]: osdmap e92: 8 total, 8 up, 8 in 2024-09-16T08:47:54.705 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6"} : dispatch 2024-09-16T08:47:54.705 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-9"} : dispatch 2024-09-16T08:47:54.706 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3145873442' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:54.706 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[30137]: from='client.32671 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:54.706 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[30137]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6"} : dispatch 2024-09-16T08:47:54.706 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:55.099 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:47:54 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:47:55.413 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:47:55.413 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-9"}]': finished 2024-09-16T08:47:55.413 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: from='client.32671 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:55.413 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6"}]': finished 2024-09-16T08:47:55.413 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: osdmap e93: 8 total, 8 up, 8 in 2024-09-16T08:47:55.413 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-9", "mode": "writeback"} : dispatch 2024-09-16T08:47:55.414 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2793451756' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:55.414 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: from='client.33685 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:55.414 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi160-76317-6", "pool2": "test-rados-api-smithi160-76317-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T08:47:55.414 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi160-76317-6", "pool2": "test-rados-api-smithi160-76317-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T08:47:55.414 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:55.414 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:55.414 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:55.414 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:55.414 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:55.414 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:55.414 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:55.415 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:47:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:47:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:47:55.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:47:55.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-9"}]': finished 2024-09-16T08:47:55.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: from='client.32671 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:55.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6"}]': finished 2024-09-16T08:47:55.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: osdmap e93: 8 total, 8 up, 8 in 2024-09-16T08:47:55.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-9", "mode": "writeback"} : dispatch 2024-09-16T08:47:55.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2793451756' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:55.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: from='client.33685 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:55.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi160-76317-6", "pool2": "test-rados-api-smithi160-76317-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T08:47:55.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi160-76317-6", "pool2": "test-rados-api-smithi160-76317-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T08:47:55.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:55.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:55.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:55.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:55.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:55.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:55.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:55.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:47:55.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-9"}]': finished 2024-09-16T08:47:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: from='client.32671 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-76317-4", "tierpool": "test-rados-api-smithi160-76317-6"}]': finished 2024-09-16T08:47:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: osdmap e93: 8 total, 8 up, 8 in 2024-09-16T08:47:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-9", "mode": "writeback"} : dispatch 2024-09-16T08:47:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2793451756' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: from='client.33685 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3312897027' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi160-76317-6", "pool2": "test-rados-api-smithi160-76317-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T08:47:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: from='client.35345 ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi160-76317-6", "pool2": "test-rados-api-smithi160-76317-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T08:47:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:47:55.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:55.994 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-09-16T08:47:55.994 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [----------] Global test environment set-up. 2024-09-16T08:47:55.994 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [----------] 4 tests from LibRadosService 2024-09-16T08:47:55.994 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-09-16T08:47:55.994 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5050 ms) 2024-09-16T08:47:55.994 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-09-16T08:47:55.995 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ OK ] LibRadosService.RegisterLate (15 ms) 2024-09-16T08:47:55.995 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-09-16T08:47:55.995 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: cluster: 2024-09-16T08:47:55.995 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: id: 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:47:55.995 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: health: HEALTH_WARN 2024-09-16T08:47:55.995 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 15 pool(s) do not have an application enabled 2024-09-16T08:47:55.995 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-16T08:47:55.995 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: services: 2024-09-16T08:47:55.995 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-09-16T08:47:55.995 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: mgr: y(active, since 90s), standbys: x 2024-09-16T08:47:55.995 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-09-16T08:47:55.995 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-09-16T08:47:55.995 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-16T08:47:55.996 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-16T08:47:55.996 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: data: 2024-09-16T08:47:55.996 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: pools: 30 pools, 852 pgs 2024-09-16T08:47:55.996 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: objects: 199 objects, 583 KiB 2024-09-16T08:47:55.996 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: usage: 234 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:47:55.996 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: pgs: 84.507% pgs unknown 2024-09-16T08:47:55.996 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 720 unknown 2024-09-16T08:47:55.996 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 132 active+clean 2024-09-16T08:47:55.996 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-16T08:47:55.996 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-16T08:47:55.996 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: cluster: 2024-09-16T08:47:55.997 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: id: 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T08:47:55.997 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: health: HEALTH_WARN 2024-09-16T08:47:55.997 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 11 pool(s) do not have an application enabled 2024-09-16T08:47:55.997 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-16T08:47:55.997 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: services: 2024-09-16T08:47:55.997 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-09-16T08:47:55.997 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: mgr: y(active, since 92s), standbys: x 2024-09-16T08:47:55.997 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-09-16T08:47:55.997 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-09-16T08:47:55.997 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-09-16T08:47:55.997 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-16T08:47:55.997 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-16T08:47:55.997 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: data: 2024-09-16T08:47:55.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: pools: 26 pools, 676 pgs 2024-09-16T08:47:55.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: objects: 257 objects, 145 MiB 2024-09-16T08:47:55.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: usage: 715 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:47:55.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: pgs: 18.935% pgs unknown 2024-09-16T08:47:55.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 14.201% pgs not active 2024-09-16T08:47:55.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 452 active+clean 2024-09-16T08:47:55.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 128 unknown 2024-09-16T08:47:55.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 96 creating+peering 2024-09-16T08:47:55.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-16T08:47:55.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: io: 2024-09-16T08:47:55.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: client: 12 MiB/s rd, 37 MiB/s wr, 256 op/s rd, 291 op/s wr 2024-09-16T08:47:55.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-16T08:47:55.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-16T08:47:55.999 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ OK ] LibRadosService.StatusFormat (2165 ms) 2024-09-16T08:47:55.999 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ RUN ] LibRadosService.Status 2024-09-16T08:47:55.999 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ OK ] LibRadosService.Status (20026 ms) 2024-09-16T08:47:55.999 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [----------] 4 tests from LibRadosService (27256 ms total) 2024-09-16T08:47:55.999 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-16T08:47:55.999 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [----------] Global test environment tear-down 2024-09-16T08:47:55.999 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [==========] 4 tests from 1 test suite ran. (27256 ms total) 2024-09-16T08:47:55.999 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ PASSED ] 4 tests. 2024-09-16T08:47:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[30137]: pgmap v87: 584 pgs: 64 unknown, 96 creating+peering, 424 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:47:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:47:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-9", "mode": "writeback"}]': finished 2024-09-16T08:47:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[30137]: from='client.33685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[30137]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi160-76317-6", "pool2": "test-rados-api-smithi160-76317-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T08:47:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[30137]: from='client.43949 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[30137]: osdmap e94: 8 total, 8 up, 8 in 2024-09-16T08:47:56.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi160-76059-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:56.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[30137]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi160-76059-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:56.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1391874651' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi160-75741-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:56.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[30137]: from='client.43928 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi160-75741-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:56.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:56.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:47:56.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:56.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[40028]: pgmap v87: 584 pgs: 64 unknown, 96 creating+peering, 424 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:47:56.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:47:56.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-9", "mode": "writeback"}]': finished 2024-09-16T08:47:56.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[40028]: from='client.33685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:56.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[40028]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi160-76317-6", "pool2": "test-rados-api-smithi160-76317-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T08:47:56.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[40028]: from='client.43949 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:56.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[40028]: osdmap e94: 8 total, 8 up, 8 in 2024-09-16T08:47:56.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi160-76059-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:56.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[40028]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi160-76059-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:56.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1391874651' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi160-75741-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:56.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[40028]: from='client.43928 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi160-75741-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:56.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:56.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:47:56.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:56 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:47:56.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:56 smithi169 ceph-mon[33465]: pgmap v87: 584 pgs: 64 unknown, 96 creating+peering, 424 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:47:56.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:56 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:47:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-9", "mode": "writeback"}]': finished 2024-09-16T08:47:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:56 smithi169 ceph-mon[33465]: from='client.33685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi160-75757-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:56 smithi169 ceph-mon[33465]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi160-76317-6", "pool2": "test-rados-api-smithi160-76317-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T08:47:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:56 smithi169 ceph-mon[33465]: from='client.43949 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:47:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:56 smithi169 ceph-mon[33465]: osdmap e94: 8 total, 8 up, 8 in 2024-09-16T08:47:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi160-76059-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:56 smithi169 ceph-mon[33465]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi160-76059-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:47:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1391874651' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi160-75741-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:56 smithi169 ceph-mon[33465]: from='client.43928 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi160-75741-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:47:57.162 INFO:tasks.workunit.client.0.smithi160.stdout:ch_notify_pp: flushed 2024-09-16T08:47:57.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3004 ms) 2024-09-16T08:47:57.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-09-16T08:47:57.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: trying... 2024-09-16T08:47:57.162 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 94060058797872 notify_id 352187318279 notifier_gid 35345 2024-09-16T08:47:57.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: timed out 2024-09-16T08:47:57.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: flushing 2024-09-16T08:47:57.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: flushed 2024-09-16T08:47:57.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3004 ms) 2024-09-16T08:47:57.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-09-16T08:47:57.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: List watches 2024-09-16T08:47:57.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: notify2 2024-09-16T08:47:57.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 94060058797872 notify_id 365072220162 notifier_gid 35345 2024-09-16T08:47:57.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: notify2 done 2024-09-16T08:47:57.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: watch_check 2024-09-16T08:47:57.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: unwatch2 2024-09-16T08:47:57.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: flushing 2024-09-16T08:47:57.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: done 2024-09-16T08:47:57.163 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3013 ms) 2024-09-16T08:47:57.164 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-09-16T08:47:57.164 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: List watches 2024-09-16T08:47:57.164 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: notify2 2024-09-16T08:47:57.164 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 94060058621168 notify_id 377957122051 notifier_gid 35345 2024-09-16T08:47:57.164 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: notify2 done 2024-09-16T08:47:57.164 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: watch_check 2024-09-16T08:47:57.164 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: unwatch2 2024-09-16T08:47:57.164 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: flushing 2024-09-16T08:47:57.164 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: done 2024-09-16T08:47:57.164 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3024 ms) 2024-09-16T08:47:57.164 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16083 ms total) 2024-09-16T08:47:57.164 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: 2024-09-16T08:47:57.164 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-09-16T08:47:57.165 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (28463 ms total) 2024-09-16T08:47:57.165 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-09-16T08:47:58.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:58 smithi160 ceph-mon[40028]: pgmap v90: 612 pgs: 224 unknown, 388 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail 2024-09-16T08:47:58.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:58 smithi160 ceph-mon[40028]: from='client.43928 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi160-75741-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:58.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:47:58.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:58 smithi160 ceph-mon[40028]: osdmap e95: 8 total, 8 up, 8 in 2024-09-16T08:47:58.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-9"} : dispatch 2024-09-16T08:47:58.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:58.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:58 smithi160 ceph-mon[40028]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:47:58.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:58 smithi160 ceph-mon[30137]: pgmap v90: 612 pgs: 224 unknown, 388 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail 2024-09-16T08:47:58.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:58 smithi160 ceph-mon[30137]: from='client.43928 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi160-75741-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:58.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:47:58.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:58 smithi160 ceph-mon[30137]: osdmap e95: 8 total, 8 up, 8 in 2024-09-16T08:47:58.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-9"} : dispatch 2024-09-16T08:47:58.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:58.452 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:58 smithi160 ceph-mon[30137]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:47:58.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:58 smithi169 ceph-mon[33465]: pgmap v90: 612 pgs: 224 unknown, 388 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail 2024-09-16T08:47:58.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:58 smithi169 ceph-mon[33465]: from='client.43928 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi160-75741-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:58.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:47:58.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:58 smithi169 ceph-mon[33465]: osdmap e95: 8 total, 8 up, 8 in 2024-09-16T08:47:58.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-9"} : dispatch 2024-09-16T08:47:58.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:58.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:58 smithi169 ceph-mon[33465]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:47:58.997 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-09-16T08:47:58.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [----------] Global test environment set-up. 2024-09-16T08:47:58.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-09-16T08:47:58.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-09-16T08:47:58.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5046 ms) 2024-09-16T08:47:58.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-09-16T08:47:58.998 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (41 ms) 2024-09-16T08:47:58.999 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-09-16T08:47:58.999 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20022 ms) 2024-09-16T08:47:58.999 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-09-16T08:47:58.999 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: attempt 0 of 20 2024-09-16T08:47:58.999 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (5119 ms) 2024-09-16T08:47:58.999 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (30228 ms total) 2024-09-16T08:47:59.000 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: 2024-09-16T08:47:59.000 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [----------] Global test environment tear-down 2024-09-16T08:47:59.000 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (30228 ms total) 2024-09-16T08:47:59.000 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-09-16T08:47:59.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:47:59.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[30137]: from='client.43949 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi160-76059-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi160-76059-10"}]': finished 2024-09-16T08:47:59.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-9"}]': finished 2024-09-16T08:47:59.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[30137]: osdmap e96: 8 total, 8 up, 8 in 2024-09-16T08:47:59.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2975669846' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi160-75757-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[30137]: from='client.36940 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi160-75757-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:59.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3649433789' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-16T08:47:59.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[30137]: from='client.36940 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi160-75757-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:59.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[30137]: osdmap e97: 8 total, 8 up, 8 in 2024-09-16T08:47:59.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1385349609' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi160-75741-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[30137]: from='client.38062 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi160-75741-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1542423339' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[30137]: from='client.38077 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:47:59.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[40028]: from='client.43949 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi160-76059-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi160-76059-10"}]': finished 2024-09-16T08:47:59.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-9"}]': finished 2024-09-16T08:47:59.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[40028]: osdmap e96: 8 total, 8 up, 8 in 2024-09-16T08:47:59.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2975669846' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi160-75757-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[40028]: from='client.36940 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi160-75757-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:59.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3649433789' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-16T08:47:59.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[40028]: from='client.36940 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi160-75757-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:59.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[40028]: osdmap e97: 8 total, 8 up, 8 in 2024-09-16T08:47:59.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1385349609' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi160-75741-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[40028]: from='client.38062 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi160-75741-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1542423339' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:47:59 smithi160 ceph-mon[40028]: from='client.38077 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:59 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:47:59.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:59 smithi169 ceph-mon[33465]: from='client.43949 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi160-76059-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi160-76059-10"}]': finished 2024-09-16T08:47:59.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-9"}]': finished 2024-09-16T08:47:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:59 smithi169 ceph-mon[33465]: osdmap e96: 8 total, 8 up, 8 in 2024-09-16T08:47:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2975669846' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi160-75757-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:59 smithi169 ceph-mon[33465]: from='client.36940 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi160-75757-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:47:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3649433789' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-16T08:47:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:59 smithi169 ceph-mon[33465]: from='client.36940 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi160-75757-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:47:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:59 smithi169 ceph-mon[33465]: osdmap e97: 8 total, 8 up, 8 in 2024-09-16T08:47:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1385349609' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi160-75741-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:59 smithi169 ceph-mon[33465]: from='client.38062 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi160-75741-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1542423339' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:47:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:47:59 smithi169 ceph-mon[33465]: from='client.38077 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:00.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:00 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-16T08:48:00.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:00 smithi169 ceph-mon[33465]: pgmap v93: 428 pgs: 40 creating+peering, 388 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:00.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:00.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:00 smithi169 ceph-mon[33465]: from='client.38062 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi160-75741-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:00 smithi169 ceph-mon[33465]: from='client.38077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:00 smithi169 ceph-mon[33465]: osdmap e98: 8 total, 8 up, 8 in 2024-09-16T08:48:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:00.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:00 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-16T08:48:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:00 smithi160 ceph-mon[40028]: pgmap v93: 428 pgs: 40 creating+peering, 388 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:00 smithi160 ceph-mon[40028]: from='client.38062 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi160-75741-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:00 smithi160 ceph-mon[40028]: from='client.38077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:00 smithi160 ceph-mon[40028]: osdmap e98: 8 total, 8 up, 8 in 2024-09-16T08:48:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:00 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-16T08:48:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:00 smithi160 ceph-mon[30137]: pgmap v93: 428 pgs: 40 creating+peering, 388 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:00.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:00.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:00 smithi160 ceph-mon[30137]: from='client.38062 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi160-75741-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:00.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:00 smithi160 ceph-mon[30137]: from='client.38077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76029-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:00.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:00 smithi160 ceph-mon[30137]: osdmap e98: 8 total, 8 up, 8 in 2024-09-16T08:48:00.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:01.171 INFO:tasks.workunit.client.0.smithi160.stdout: misc: Running main() from gmock_main.cc 2024-09-16T08:48:01.172 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-09-16T08:48:01.172 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [----------] Global test environment set-up. 2024-09-16T08:48:01.172 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-09-16T08:48:01.172 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-09-16T08:48:01.172 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.Version (1238 ms) 2024-09-16T08:48:01.172 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-09-16T08:48:01.172 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (1999 ms) 2024-09-16T08:48:01.172 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-09-16T08:48:01.172 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.LongName (3038 ms) 2024-09-16T08:48:01.172 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-09-16T08:48:01.173 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (3021 ms) 2024-09-16T08:48:01.173 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-09-16T08:48:01.173 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (2886 ms) 2024-09-16T08:48:01.173 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-09-16T08:48:01.173 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (3006 ms) 2024-09-16T08:48:01.173 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-09-16T08:48:01.173 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.Exec (3029 ms) 2024-09-16T08:48:01.173 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-09-16T08:48:01.173 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (3020 ms) 2024-09-16T08:48:01.173 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-09-16T08:48:01.173 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (3013 ms) 2024-09-16T08:48:01.173 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-09-16T08:48:01.173 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.BigObject (2968 ms) 2024-09-16T08:48:01.173 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-09-16T08:48:01.174 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (1991 ms) 2024-09-16T08:48:01.174 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-09-16T08:48:01.174 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (3004 ms) 2024-09-16T08:48:01.174 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [----------] 12 tests from NeoRadosMisc (32213 ms total) 2024-09-16T08:48:01.174 INFO:tasks.workunit.client.0.smithi160.stdout: misc: 2024-09-16T08:48:01.174 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [----------] Global test environment tear-down 2024-09-16T08:48:01.174 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [==========] 12 tests from 1 test suite ran. (32213 ms total) 2024-09-16T08:48:01.174 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ PASSED ] 12 tests. 2024-09-16T08:48:01.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:01.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:01.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:01 smithi169 ceph-mon[33465]: osdmap e99: 8 total, 8 up, 8 in 2024-09-16T08:48:01.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1366978115' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi160-75757-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:01.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1542423339' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache", "force_nonempty":""} : dispatch 2024-09-16T08:48:01.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:01 smithi169 ceph-mon[33465]: from='client.40291 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi160-75757-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:01.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:01 smithi169 ceph-mon[33465]: from='client.38077 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache", "force_nonempty":""} : dispatch 2024-09-16T08:48:01.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:01 smithi160 ceph-mon[30137]: osdmap e99: 8 total, 8 up, 8 in 2024-09-16T08:48:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1366978115' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi160-75757-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1542423339' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache", "force_nonempty":""} : dispatch 2024-09-16T08:48:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:01 smithi160 ceph-mon[30137]: from='client.40291 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi160-75757-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:01 smithi160 ceph-mon[30137]: from='client.38077 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache", "force_nonempty":""} : dispatch 2024-09-16T08:48:01.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:01.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:01.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:01 smithi160 ceph-mon[40028]: osdmap e99: 8 total, 8 up, 8 in 2024-09-16T08:48:01.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1366978115' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi160-75757-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:01.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1542423339' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache", "force_nonempty":""} : dispatch 2024-09-16T08:48:01.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:01 smithi160 ceph-mon[40028]: from='client.40291 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi160-75757-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:01.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:01 smithi160 ceph-mon[40028]: from='client.38077 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache", "force_nonempty":""} : dispatch 2024-09-16T08:48:02.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:02 smithi169 ceph-mon[33465]: pgmap v96: 556 pgs: 224 unknown, 8 creating+peering, 324 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 5 op/s 2024-09-16T08:48:02.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:02 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:02.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:02 smithi169 ceph-mon[33465]: from='client.40291 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi160-75757-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:02.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:02 smithi169 ceph-mon[33465]: from='client.38077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache", "force_nonempty":""}]': finished 2024-09-16T08:48:02.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:02 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1542423339' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76029-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:02.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:02 smithi169 ceph-mon[33465]: osdmap e100: 8 total, 8 up, 8 in 2024-09-16T08:48:02.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:02 smithi169 ceph-mon[33465]: from='client.38077 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76029-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:02.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:02 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4041972937' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi160-75741-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:02.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:02 smithi169 ceph-mon[33465]: from='client.41323 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi160-75741-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:02.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:02 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:48:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[30137]: pgmap v96: 556 pgs: 224 unknown, 8 creating+peering, 324 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 5 op/s 2024-09-16T08:48:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[30137]: from='client.40291 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi160-75757-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:02.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[30137]: from='client.38077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache", "force_nonempty":""}]': finished 2024-09-16T08:48:02.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1542423339' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76029-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:02.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[30137]: osdmap e100: 8 total, 8 up, 8 in 2024-09-16T08:48:02.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[30137]: from='client.38077 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76029-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:02.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4041972937' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi160-75741-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:02.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[30137]: from='client.41323 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi160-75741-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:02.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:48:02.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[40028]: pgmap v96: 556 pgs: 224 unknown, 8 creating+peering, 324 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 5 op/s 2024-09-16T08:48:02.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:02.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[40028]: from='client.40291 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi160-75757-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:02.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[40028]: from='client.38077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache", "force_nonempty":""}]': finished 2024-09-16T08:48:02.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1542423339' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76029-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:02.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[40028]: osdmap e100: 8 total, 8 up, 8 in 2024-09-16T08:48:02.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[40028]: from='client.38077 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76029-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:02.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4041972937' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi160-75741-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:02.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[40028]: from='client.41323 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi160-75741-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:02.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:02 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:48:03.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:03.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:03 smithi169 ceph-mon[33465]: from='client.38077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76029-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:03.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:03 smithi169 ceph-mon[33465]: from='client.41323 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi160-75741-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:03.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:48:03.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:03 smithi169 ceph-mon[33465]: osdmap e101: 8 total, 8 up, 8 in 2024-09-16T08:48:03.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-11"} : dispatch 2024-09-16T08:48:03.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1542423339' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache"} : dispatch 2024-09-16T08:48:03.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:03 smithi169 ceph-mon[33465]: from='client.38077 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache"} : dispatch 2024-09-16T08:48:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:03 smithi160 ceph-mon[40028]: from='client.38077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76029-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:03 smithi160 ceph-mon[40028]: from='client.41323 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi160-75741-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:48:03.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:03 smithi160 ceph-mon[40028]: osdmap e101: 8 total, 8 up, 8 in 2024-09-16T08:48:03.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-11"} : dispatch 2024-09-16T08:48:03.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1542423339' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache"} : dispatch 2024-09-16T08:48:03.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:03 smithi160 ceph-mon[40028]: from='client.38077 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache"} : dispatch 2024-09-16T08:48:03.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:03.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:03 smithi160 ceph-mon[30137]: from='client.38077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76029-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:03.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:03 smithi160 ceph-mon[30137]: from='client.41323 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi160-75741-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:03.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:48:03.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:03 smithi160 ceph-mon[30137]: osdmap e101: 8 total, 8 up, 8 in 2024-09-16T08:48:03.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-11"} : dispatch 2024-09-16T08:48:03.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1542423339' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache"} : dispatch 2024-09-16T08:48:03.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:03 smithi160 ceph-mon[30137]: from='client.38077 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache"} : dispatch 2024-09-16T08:48:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: pgmap v99: 588 pgs: 256 unknown, 8 creating+peering, 324 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:48:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.0"} : dispatch 2024-09-16T08:48:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-16T08:48:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.1"} : dispatch 2024-09-16T08:48:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-16T08:48:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.2"} : dispatch 2024-09-16T08:48:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-16T08:48:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.3"} : dispatch 2024-09-16T08:48:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-16T08:48:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.4"} : dispatch 2024-09-16T08:48:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-16T08:48:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.5"} : dispatch 2024-09-16T08:48:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-16T08:48:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.6"} : dispatch 2024-09-16T08:48:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-16T08:48:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.7"} : dispatch 2024-09-16T08:48:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-16T08:48:04.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.8"} : dispatch 2024-09-16T08:48:04.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-16T08:48:04.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.9"} : dispatch 2024-09-16T08:48:04.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-16T08:48:04.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-11"}]': finished 2024-09-16T08:48:04.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.38077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache"}]': finished 2024-09-16T08:48:04.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: osdmap e102: 8 total, 8 up, 8 in 2024-09-16T08:48:04.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-11", "mode": "writeback"} : dispatch 2024-09-16T08:48:04.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2697223167' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi160-75757-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:04.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.43657 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi160-75757-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:04.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: pgmap v99: 588 pgs: 256 unknown, 8 creating+peering, 324 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:04.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:48:04.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.0"} : dispatch 2024-09-16T08:48:04.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-16T08:48:04.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.1"} : dispatch 2024-09-16T08:48:04.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-16T08:48:04.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.2"} : dispatch 2024-09-16T08:48:04.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-16T08:48:04.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.3"} : dispatch 2024-09-16T08:48:04.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-16T08:48:04.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.4"} : dispatch 2024-09-16T08:48:04.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-16T08:48:04.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.5"} : dispatch 2024-09-16T08:48:04.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-16T08:48:04.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.6"} : dispatch 2024-09-16T08:48:04.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-16T08:48:04.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.7"} : dispatch 2024-09-16T08:48:04.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-16T08:48:04.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.8"} : dispatch 2024-09-16T08:48:04.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-16T08:48:04.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.9"} : dispatch 2024-09-16T08:48:04.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-16T08:48:04.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-11"}]': finished 2024-09-16T08:48:04.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.38077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache"}]': finished 2024-09-16T08:48:04.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: osdmap e102: 8 total, 8 up, 8 in 2024-09-16T08:48:04.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-11", "mode": "writeback"} : dispatch 2024-09-16T08:48:04.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2697223167' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi160-75757-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:04.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.43657 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi160-75757-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:04.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:04.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: pgmap v99: 588 pgs: 256 unknown, 8 creating+peering, 324 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:04.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:04.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:04.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:04.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:48:04.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.0"} : dispatch 2024-09-16T08:48:04.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-16T08:48:04.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.1"} : dispatch 2024-09-16T08:48:04.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-16T08:48:04.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.2"} : dispatch 2024-09-16T08:48:04.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-16T08:48:04.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.3"} : dispatch 2024-09-16T08:48:04.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-16T08:48:04.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.4"} : dispatch 2024-09-16T08:48:04.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-16T08:48:04.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.5"} : dispatch 2024-09-16T08:48:04.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-16T08:48:04.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.6"} : dispatch 2024-09-16T08:48:04.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-16T08:48:04.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.7"} : dispatch 2024-09-16T08:48:04.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-16T08:48:04.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.8"} : dispatch 2024-09-16T08:48:04.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-16T08:48:04.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.9"} : dispatch 2024-09-16T08:48:04.704 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.169:0/3422783404' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-16T08:48:04.704 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-11"}]': finished 2024-09-16T08:48:04.704 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.38077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76029-10", "tierpool":"test-rados-api-smithi160-76029-10-cache"}]': finished 2024-09-16T08:48:04.704 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: osdmap e102: 8 total, 8 up, 8 in 2024-09-16T08:48:04.704 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-11", "mode": "writeback"} : dispatch 2024-09-16T08:48:04.704 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2697223167' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi160-75757-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:04.704 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.43657 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi160-75757-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:04.704 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:05.099 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:48:04 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:48:05.411 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[30137]: 17.5 deep-scrub starts 2024-09-16T08:48:05.411 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[30137]: 17.5 deep-scrub ok 2024-09-16T08:48:05.412 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[30137]: 17.1 deep-scrub starts 2024-09-16T08:48:05.412 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[30137]: 17.1 deep-scrub ok 2024-09-16T08:48:05.412 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[30137]: 17.0 deep-scrub starts 2024-09-16T08:48:05.412 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[30137]: 17.0 deep-scrub ok 2024-09-16T08:48:05.412 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[30137]: 17.9 deep-scrub starts 2024-09-16T08:48:05.412 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[30137]: 17.9 deep-scrub ok 2024-09-16T08:48:05.412 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[30137]: 17.2 deep-scrub starts 2024-09-16T08:48:05.412 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:48:05.412 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-11", "mode": "writeback"}]': finished 2024-09-16T08:48:05.412 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[30137]: from='client.43657 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi160-75757-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:05.412 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[30137]: osdmap e103: 8 total, 8 up, 8 in 2024-09-16T08:48:05.412 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2493859447' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi160-75741-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:05.413 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[30137]: from='client.44686 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi160-75741-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:05.413 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:05.413 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:48:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:48:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:48:05.413 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[40028]: 17.5 deep-scrub starts 2024-09-16T08:48:05.413 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[40028]: 17.5 deep-scrub ok 2024-09-16T08:48:05.413 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[40028]: 17.1 deep-scrub starts 2024-09-16T08:48:05.413 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[40028]: 17.1 deep-scrub ok 2024-09-16T08:48:05.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[40028]: 17.0 deep-scrub starts 2024-09-16T08:48:05.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[40028]: 17.0 deep-scrub ok 2024-09-16T08:48:05.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[40028]: 17.9 deep-scrub starts 2024-09-16T08:48:05.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[40028]: 17.9 deep-scrub ok 2024-09-16T08:48:05.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[40028]: 17.2 deep-scrub starts 2024-09-16T08:48:05.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:48:05.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-11", "mode": "writeback"}]': finished 2024-09-16T08:48:05.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[40028]: from='client.43657 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi160-75757-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:05.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[40028]: osdmap e103: 8 total, 8 up, 8 in 2024-09-16T08:48:05.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2493859447' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi160-75741-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:05.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[40028]: from='client.44686 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi160-75741-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:05.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:05.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:05 smithi169 ceph-mon[33465]: 17.5 deep-scrub starts 2024-09-16T08:48:05.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:05 smithi169 ceph-mon[33465]: 17.5 deep-scrub ok 2024-09-16T08:48:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:05 smithi169 ceph-mon[33465]: 17.1 deep-scrub starts 2024-09-16T08:48:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:05 smithi169 ceph-mon[33465]: 17.1 deep-scrub ok 2024-09-16T08:48:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:05 smithi169 ceph-mon[33465]: 17.0 deep-scrub starts 2024-09-16T08:48:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:05 smithi169 ceph-mon[33465]: 17.0 deep-scrub ok 2024-09-16T08:48:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:05 smithi169 ceph-mon[33465]: 17.9 deep-scrub starts 2024-09-16T08:48:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:05 smithi169 ceph-mon[33465]: 17.9 deep-scrub ok 2024-09-16T08:48:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:05 smithi169 ceph-mon[33465]: 17.2 deep-scrub starts 2024-09-16T08:48:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:05 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:48:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-11", "mode": "writeback"}]': finished 2024-09-16T08:48:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:05 smithi169 ceph-mon[33465]: from='client.43657 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi160-75757-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:05 smithi169 ceph-mon[33465]: osdmap e103: 8 total, 8 up, 8 in 2024-09-16T08:48:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2493859447' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi160-75741-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:05 smithi169 ceph-mon[33465]: from='client.44686 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi160-75741-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:05.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:06.196 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: Running main() from gmock_main.cc 2024-09-16T08:48:06.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-09-16T08:48:06.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [----------] Global test environment set-up. 2024-09-16T08:48:06.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-09-16T08:48:06.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-09-16T08:48:06.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolList (2575 ms) 2024-09-16T08:48:06.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-09-16T08:48:06.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (3003 ms) 2024-09-16T08:48:06.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-09-16T08:48:06.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (3033 ms) 2024-09-16T08:48:06.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-09-16T08:48:06.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (2903 ms) 2024-09-16T08:48:06.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-09-16T08:48:06.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3004 ms) 2024-09-16T08:48:06.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-09-16T08:48:06.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5047 ms) 2024-09-16T08:48:06.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-09-16T08:48:06.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5009 ms) 2024-09-16T08:48:06.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-09-16T08:48:06.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (4948 ms) 2024-09-16T08:48:06.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-09-16T08:48:06.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (8055 ms) 2024-09-16T08:48:06.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [----------] 9 tests from LibRadosPools (37577 ms total) 2024-09-16T08:48:06.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: 2024-09-16T08:48:06.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [----------] Global test environment tear-down 2024-09-16T08:48:06.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (37577 ms total) 2024-09-16T08:48:06.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ PASSED ] 9 tests. 2024-09-16T08:48:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[30137]: 17.2 deep-scrub ok 2024-09-16T08:48:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[30137]: 17.6 deep-scrub starts 2024-09-16T08:48:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[30137]: 17.6 deep-scrub ok 2024-09-16T08:48:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[30137]: 17.4 deep-scrub starts 2024-09-16T08:48:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[30137]: 17.4 deep-scrub ok 2024-09-16T08:48:06.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[30137]: pgmap v102: 492 pgs: 32 unknown, 460 active+clean; 217 MiB data, 977 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 18 MiB/s wr, 8 op/s 2024-09-16T08:48:06.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[30137]: 17.8 deep-scrub starts 2024-09-16T08:48:06.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[30137]: 17.8 deep-scrub ok 2024-09-16T08:48:06.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[30137]: 17.3 deep-scrub starts 2024-09-16T08:48:06.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[30137]: 17.3 deep-scrub ok 2024-09-16T08:48:06.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[30137]: from='client.44686 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi160-75741-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:06.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[30137]: osdmap e104: 8 total, 8 up, 8 in 2024-09-16T08:48:06.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:06.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:48:06.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[40028]: 17.2 deep-scrub ok 2024-09-16T08:48:06.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[40028]: 17.6 deep-scrub starts 2024-09-16T08:48:06.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[40028]: 17.6 deep-scrub ok 2024-09-16T08:48:06.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:06.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[40028]: 17.4 deep-scrub starts 2024-09-16T08:48:06.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[40028]: 17.4 deep-scrub ok 2024-09-16T08:48:06.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[40028]: pgmap v102: 492 pgs: 32 unknown, 460 active+clean; 217 MiB data, 977 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 18 MiB/s wr, 8 op/s 2024-09-16T08:48:06.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[40028]: 17.8 deep-scrub starts 2024-09-16T08:48:06.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[40028]: 17.8 deep-scrub ok 2024-09-16T08:48:06.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[40028]: 17.3 deep-scrub starts 2024-09-16T08:48:06.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[40028]: 17.3 deep-scrub ok 2024-09-16T08:48:06.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[40028]: from='client.44686 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi160-75741-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:06.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[40028]: osdmap e104: 8 total, 8 up, 8 in 2024-09-16T08:48:06.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:06.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:06 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:48:06.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:06 smithi169 ceph-mon[33465]: 17.2 deep-scrub ok 2024-09-16T08:48:06.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:06 smithi169 ceph-mon[33465]: 17.6 deep-scrub starts 2024-09-16T08:48:06.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:06 smithi169 ceph-mon[33465]: 17.6 deep-scrub ok 2024-09-16T08:48:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:06 smithi169 ceph-mon[33465]: 17.4 deep-scrub starts 2024-09-16T08:48:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:06 smithi169 ceph-mon[33465]: 17.4 deep-scrub ok 2024-09-16T08:48:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:06 smithi169 ceph-mon[33465]: pgmap v102: 492 pgs: 32 unknown, 460 active+clean; 217 MiB data, 977 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 18 MiB/s wr, 8 op/s 2024-09-16T08:48:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:06 smithi169 ceph-mon[33465]: 17.8 deep-scrub starts 2024-09-16T08:48:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:06 smithi169 ceph-mon[33465]: 17.8 deep-scrub ok 2024-09-16T08:48:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:06 smithi169 ceph-mon[33465]: 17.3 deep-scrub starts 2024-09-16T08:48:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:06 smithi169 ceph-mon[33465]: 17.3 deep-scrub ok 2024-09-16T08:48:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:06 smithi169 ceph-mon[33465]: from='client.44686 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi160-75741-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:06 smithi169 ceph-mon[33465]: osdmap e104: 8 total, 8 up, 8 in 2024-09-16T08:48:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:06 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:06 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:48:07.178 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-09-16T08:48:07.178 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] Global test environment set-up. 2024-09-16T08:48:07.178 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-09-16T08:48:07.178 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-09-16T08:48:07.178 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-09-16T08:48:07.178 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-16T08:48:07.178 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: 2024-09-16T08:48:07.178 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-09-16T08:48:07.178 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-09-16T08:48:07.178 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-09-16T08:48:07.178 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: 2024-09-16T08:47:28.655+0000 7fc690033780 -1 failed for service _ceph-mon._tcp 2024-09-16T08:48:07.179 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: 2024-09-16T08:47:28.655+0000 7fc690033780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-09-16T08:48:07.179 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (59 ms) 2024-09-16T08:48:07.179 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-09-16T08:48:07.179 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (5012 ms) 2024-09-16T08:48:07.179 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (5071 ms total) 2024-09-16T08:48:07.179 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: 2024-09-16T08:48:07.179 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-09-16T08:48:07.179 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-09-16T08:48:07.179 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: started 0x55681e1ae6b0 2024-09-16T08:48:07.179 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: started 0x55681e2e9f60 2024-09-16T08:48:07.179 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: started 2 aios 2024-09-16T08:48:07.179 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: waiting 0x55681e1ae6b0 2024-09-16T08:48:07.179 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: waiting 0x55681e2e9f60 2024-09-16T08:48:07.179 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: done. 2024-09-16T08:48:07.180 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (5448 ms) 2024-09-16T08:48:07.180 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (5448 ms total) 2024-09-16T08:48:07.180 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: 2024-09-16T08:48:07.180 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-09-16T08:48:07.180 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-09-16T08:48:07.180 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-09-16T08:48:07.180 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-09-16T08:48:07.180 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMisc.Exec (62 ms) 2024-09-16T08:48:07.180 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-09-16T08:48:07.180 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (11 ms) 2024-09-16T08:48:07.180 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-09-16T08:48:07.180 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (4 ms) 2024-09-16T08:48:07.180 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-09-16T08:48:07.181 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMisc.Applications (4921 ms) 2024-09-16T08:48:07.181 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-09-16T08:48:07.181 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-09-16T08:48:07.181 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-09-16T08:48:07.181 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-09-16T08:48:07.181 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-09-16T08:48:07.181 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (20845 ms) 2024-09-16T08:48:07.181 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] 8 tests from LibRadosMisc (25843 ms total) 2024-09-16T08:48:07.181 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: 2024-09-16T08:48:07.181 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] Global test environment tear-down 2024-09-16T08:48:07.181 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (38559 ms total) 2024-09-16T08:48:07.181 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ PASSED ] 12 tests. 2024-09-16T08:48:07.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:07 smithi160 ceph-mon[30137]: 17.7 deep-scrub starts 2024-09-16T08:48:07.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:07 smithi160 ceph-mon[30137]: 17.7 deep-scrub ok 2024-09-16T08:48:07.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:07 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:48:07.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:07 smithi160 ceph-mon[30137]: osdmap e105: 8 total, 8 up, 8 in 2024-09-16T08:48:07.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:07 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-11"} : dispatch 2024-09-16T08:48:07.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:07 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1189011412' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:07.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:07 smithi160 ceph-mon[30137]: from='client.46849 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:07.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:07 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:07.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:07 smithi160 ceph-mon[40028]: 17.7 deep-scrub starts 2024-09-16T08:48:07.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:07 smithi160 ceph-mon[40028]: 17.7 deep-scrub ok 2024-09-16T08:48:07.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:07 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:48:07.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:07 smithi160 ceph-mon[40028]: osdmap e105: 8 total, 8 up, 8 in 2024-09-16T08:48:07.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:07 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-11"} : dispatch 2024-09-16T08:48:07.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:07 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1189011412' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:07.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:07 smithi160 ceph-mon[40028]: from='client.46849 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:07.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:07 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:07.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:07 smithi169 ceph-mon[33465]: 17.7 deep-scrub starts 2024-09-16T08:48:07.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:07 smithi169 ceph-mon[33465]: 17.7 deep-scrub ok 2024-09-16T08:48:07.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:07 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:48:07.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:07 smithi169 ceph-mon[33465]: osdmap e105: 8 total, 8 up, 8 in 2024-09-16T08:48:07.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:07 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-11"} : dispatch 2024-09-16T08:48:07.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:07 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1189011412' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:07.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:07 smithi169 ceph-mon[33465]: from='client.46849 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:07.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:07 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:08.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[30137]: pgmap v105: 492 pgs: 128 unknown, 364 active+clean; 217 MiB data, 977 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-09-16T08:48:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:48:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-11"}]': finished 2024-09-16T08:48:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[30137]: from='client.46849 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[30137]: osdmap e106: 8 total, 8 up, 8 in 2024-09-16T08:48:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:48:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/305263714' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValue_smithi160-75741-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[30137]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:48:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[30137]: from='client.56666 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValue_smithi160-75741-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[40028]: pgmap v105: 492 pgs: 128 unknown, 364 active+clean; 217 MiB data, 977 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-09-16T08:48:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:48:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-11"}]': finished 2024-09-16T08:48:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[40028]: from='client.46849 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[40028]: osdmap e106: 8 total, 8 up, 8 in 2024-09-16T08:48:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:48:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/305263714' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValue_smithi160-75741-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[40028]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:48:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[40028]: from='client.56666 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValue_smithi160-75741-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:08.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:08 smithi169 ceph-mon[33465]: pgmap v105: 492 pgs: 128 unknown, 364 active+clean; 217 MiB data, 977 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-09-16T08:48:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:08 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:48:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-11"}]': finished 2024-09-16T08:48:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:08 smithi169 ceph-mon[33465]: from='client.46849 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:08 smithi169 ceph-mon[33465]: osdmap e106: 8 total, 8 up, 8 in 2024-09-16T08:48:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:48:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/305263714' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValue_smithi160-75741-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:08 smithi169 ceph-mon[33465]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:48:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:08 smithi169 ceph-mon[33465]: from='client.56666 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValue_smithi160-75741-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:09.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:09 smithi160 ceph-mon[40028]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:09.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:09 smithi160 ceph-mon[40028]: from='client.43949 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10"}]': finished 2024-09-16T08:48:09.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:09 smithi160 ceph-mon[40028]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi160-75741-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:09.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:09 smithi160 ceph-mon[40028]: osdmap e107: 8 total, 8 up, 8 in 2024-09-16T08:48:09.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:48:09.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:09 smithi160 ceph-mon[40028]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:48:09.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:09.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:09 smithi160 ceph-mon[30137]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:09.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:09 smithi160 ceph-mon[30137]: from='client.43949 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10"}]': finished 2024-09-16T08:48:09.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:09 smithi160 ceph-mon[30137]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi160-75741-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:09.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:09 smithi160 ceph-mon[30137]: osdmap e107: 8 total, 8 up, 8 in 2024-09-16T08:48:09.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:48:09.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:09 smithi160 ceph-mon[30137]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:48:09.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:09.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:09 smithi169 ceph-mon[33465]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:09 smithi169 ceph-mon[33465]: from='client.43949 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi160-76059-10"}]': finished 2024-09-16T08:48:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:09 smithi169 ceph-mon[33465]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi160-75741-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:09 smithi169 ceph-mon[33465]: osdmap e107: 8 total, 8 up, 8 in 2024-09-16T08:48:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/289841238' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:48:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:09 smithi169 ceph-mon[33465]: from='client.43949 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi160-76059-10"} : dispatch 2024-09-16T08:48:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:10.126 INFO:tasks.workunit.client.0.smithi160.stdout: io: Running main() from gmock_main.cc 2024-09-16T08:48:10.126 INFO:tasks.workunit.client.0.smithi160.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-09-16T08:48:10.126 INFO:tasks.workunit.client.0.smithi160.stdout: io: [----------] Global test environment set-up. 2024-09-16T08:48:10.126 INFO:tasks.workunit.client.0.smithi160.stdout: io: [----------] 14 tests from NeoRadosIo 2024-09-16T08:48:10.126 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-09-16T08:48:10.126 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.Limits (2300 ms) 2024-09-16T08:48:10.126 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-09-16T08:48:10.126 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (3019 ms) 2024-09-16T08:48:10.126 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-09-16T08:48:10.126 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.ReadOp (3022 ms) 2024-09-16T08:48:10.126 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-09-16T08:48:10.127 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.SparseRead (2899 ms) 2024-09-16T08:48:10.127 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-09-16T08:48:10.127 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.RoundTrip (3012 ms) 2024-09-16T08:48:10.127 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-09-16T08:48:10.127 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (3001 ms) 2024-09-16T08:48:10.127 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-09-16T08:48:10.127 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (3035 ms) 2024-09-16T08:48:10.127 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-09-16T08:48:10.127 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (3039 ms) 2024-09-16T08:48:10.127 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-09-16T08:48:10.127 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (2956 ms) 2024-09-16T08:48:10.127 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-09-16T08:48:10.127 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.Trunc (3018 ms) 2024-09-16T08:48:10.127 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-09-16T08:48:10.128 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.Remove (2975 ms) 2024-09-16T08:48:10.128 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-09-16T08:48:10.128 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (3044 ms) 2024-09-16T08:48:10.128 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-09-16T08:48:10.128 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.RmXattr (2975 ms) 2024-09-16T08:48:10.128 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-09-16T08:48:10.128 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.GetXattrs (2955 ms) 2024-09-16T08:48:10.128 INFO:tasks.workunit.client.0.smithi160.stdout: io: [----------] 14 tests from NeoRadosIo (41250 ms total) 2024-09-16T08:48:10.128 INFO:tasks.workunit.client.0.smithi160.stdout: io: 2024-09-16T08:48:10.128 INFO:tasks.workunit.client.0.smithi160.stdout: io: [----------] Global test environment tear-down 2024-09-16T08:48:10.128 INFO:tasks.workunit.client.0.smithi160.stdout: io: [==========] 14 tests from 1 test suite ran. (41250 ms total) 2024-09-16T08:48:10.128 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ PASSED ] 14 tests. 2024-09-16T08:48:10.130 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: Running main() from gmock_main.cc 2024-09-16T08:48:10.130 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-09-16T08:48:10.130 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [----------] Global test environment set-up. 2024-09-16T08:48:10.130 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-09-16T08:48:10.130 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-09-16T08:48:10.130 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (2240 ms) 2024-09-16T08:48:10.130 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-09-16T08:48:10.130 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (3007 ms) 2024-09-16T08:48:10.130 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-09-16T08:48:10.130 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (3029 ms) 2024-09-16T08:48:10.131 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-09-16T08:48:10.131 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (2898 ms) 2024-09-16T08:48:10.131 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-09-16T08:48:10.131 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (3010 ms) 2024-09-16T08:48:10.131 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-09-16T08:48:10.131 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (2999 ms) 2024-09-16T08:48:10.131 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-09-16T08:48:10.131 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (3043 ms) 2024-09-16T08:48:10.131 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-09-16T08:48:10.131 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (3015 ms) 2024-09-16T08:48:10.131 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-09-16T08:48:10.131 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (2967 ms) 2024-09-16T08:48:10.131 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-09-16T08:48:10.131 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (2998 ms) 2024-09-16T08:48:10.132 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-09-16T08:48:10.132 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (2983 ms) 2024-09-16T08:48:10.132 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-09-16T08:48:10.132 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (3033 ms) 2024-09-16T08:48:10.132 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-09-16T08:48:10.132 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (3002 ms) 2024-09-16T08:48:10.132 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-09-16T08:48:10.132 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (2963 ms) 2024-09-16T08:48:10.132 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (41189 ms total) 2024-09-16T08:48:10.132 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: 2024-09-16T08:48:10.132 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [----------] Global test environment tear-down 2024-09-16T08:48:10.132 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (41189 ms total) 2024-09-16T08:48:10.132 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ PASSED ] 14 tests. 2024-09-16T08:48:10.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[30137]: pgmap v108: 484 pgs: 96 creating+peering, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-16T08:48:10.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[30137]: from='client.43949 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi160-76059-10"}]': finished 2024-09-16T08:48:10.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[30137]: osdmap e108: 8 total, 8 up, 8 in 2024-09-16T08:48:10.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/47128689' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi160-75757-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:10.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:10.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[30137]: from='client.56675 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi160-75757-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:10.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:10.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[30137]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:10.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:10.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[30137]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:10.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:10.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[30137]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:10.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:10.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[40028]: pgmap v108: 484 pgs: 96 creating+peering, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-16T08:48:10.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[40028]: from='client.43949 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi160-76059-10"}]': finished 2024-09-16T08:48:10.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[40028]: osdmap e108: 8 total, 8 up, 8 in 2024-09-16T08:48:10.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/47128689' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi160-75757-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:10.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:10.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[40028]: from='client.56675 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi160-75757-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:10.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:10.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[40028]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:10.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:10.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[40028]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:10.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:10.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[40028]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:10.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:10 smithi169 ceph-mon[33465]: pgmap v108: 484 pgs: 96 creating+peering, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-16T08:48:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:10 smithi169 ceph-mon[33465]: from='client.43949 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi160-76059-10"}]': finished 2024-09-16T08:48:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:10 smithi169 ceph-mon[33465]: osdmap e108: 8 total, 8 up, 8 in 2024-09-16T08:48:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/47128689' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi160-75757-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:10 smithi169 ceph-mon[33465]: from='client.56675 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi160-75757-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:10 smithi169 ceph-mon[33465]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:10.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:10 smithi169 ceph-mon[33465]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:10.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:10.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:10 smithi169 ceph-mon[33465]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:10.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:12.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[40028]: pgmap v111: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:48:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[40028]: from='client.56675 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi160-75757-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[40028]: from='client.56681 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[40028]: osdmap e109: 8 total, 8 up, 8 in 2024-09-16T08:48:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[40028]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2353463368' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi160-75741-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[40028]: from='client.47836 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi160-75741-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:12.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:12.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[30137]: pgmap v111: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:48:12.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:12.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[30137]: from='client.56675 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi160-75757-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:12.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[30137]: from='client.56681 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:12.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[30137]: osdmap e109: 8 total, 8 up, 8 in 2024-09-16T08:48:12.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:12.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[30137]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:12.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2353463368' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi160-75741-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:12.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[30137]: from='client.47836 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi160-75741-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:12.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:12.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:12 smithi169 ceph-mon[33465]: pgmap v111: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:48:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:12 smithi169 ceph-mon[33465]: from='client.56675 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi160-75757-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:12 smithi169 ceph-mon[33465]: from='client.56681 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:12 smithi169 ceph-mon[33465]: osdmap e109: 8 total, 8 up, 8 in 2024-09-16T08:48:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:12 smithi169 ceph-mon[33465]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2353463368' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi160-75741-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:12 smithi169 ceph-mon[33465]: from='client.47836 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi160-75741-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:13.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:13 smithi160 ceph-mon[30137]: from='client.47836 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi160-75741-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:13.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:13 smithi160 ceph-mon[30137]: osdmap e110: 8 total, 8 up, 8 in 2024-09-16T08:48:13.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:13.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:13 smithi160 ceph-mon[30137]: from='client.56681 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"}]': finished 2024-09-16T08:48:13.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:13 smithi160 ceph-mon[30137]: osdmap e111: 8 total, 8 up, 8 in 2024-09-16T08:48:13.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2527997221' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi160-75757-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:13.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:13 smithi160 ceph-mon[30137]: from='client.47842 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi160-75757-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:13.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:13 smithi160 ceph-mon[40028]: from='client.47836 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi160-75741-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:13.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:13 smithi160 ceph-mon[40028]: osdmap e110: 8 total, 8 up, 8 in 2024-09-16T08:48:13.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:13.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:13 smithi160 ceph-mon[40028]: from='client.56681 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"}]': finished 2024-09-16T08:48:13.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:13 smithi160 ceph-mon[40028]: osdmap e111: 8 total, 8 up, 8 in 2024-09-16T08:48:13.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2527997221' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi160-75757-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:13.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:13 smithi160 ceph-mon[40028]: from='client.47842 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi160-75757-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:13.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:13 smithi169 ceph-mon[33465]: from='client.47836 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi160-75741-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:13.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:13 smithi169 ceph-mon[33465]: osdmap e110: 8 total, 8 up, 8 in 2024-09-16T08:48:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:13 smithi169 ceph-mon[33465]: from='client.56681 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi160-76059-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"}]': finished 2024-09-16T08:48:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:13 smithi169 ceph-mon[33465]: osdmap e111: 8 total, 8 up, 8 in 2024-09-16T08:48:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2527997221' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi160-75757-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:13 smithi169 ceph-mon[33465]: from='client.47842 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi160-75757-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:14.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:14 smithi160 ceph-mon[30137]: pgmap v114: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:14.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:14.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:14 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:48:14.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:14 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:14.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:14 smithi160 ceph-mon[30137]: from='client.47842 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi160-75757-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:14.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:14 smithi160 ceph-mon[30137]: osdmap e112: 8 total, 8 up, 8 in 2024-09-16T08:48:14.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1773519277' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi160-75741-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:14 smithi160 ceph-mon[40028]: pgmap v114: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:14 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:14 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:48:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:14 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:14 smithi160 ceph-mon[40028]: from='client.47842 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi160-75757-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:14.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:14 smithi160 ceph-mon[40028]: osdmap e112: 8 total, 8 up, 8 in 2024-09-16T08:48:14.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:14 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1773519277' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi160-75741-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:14.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:14 smithi169 ceph-mon[33465]: pgmap v114: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:14.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:14 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:14 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:48:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:14 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:14 smithi169 ceph-mon[33465]: from='client.47842 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi160-75757-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:14 smithi169 ceph-mon[33465]: osdmap e112: 8 total, 8 up, 8 in 2024-09-16T08:48:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:14 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1773519277' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi160-75741-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:15.085 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:48:14 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:48:15.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:15 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:15.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:15 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1773519277' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi160-75741-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:15.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:15 smithi169 ceph-mon[33465]: osdmap e113: 8 total, 8 up, 8 in 2024-09-16T08:48:15.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:15 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:15.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:15 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1773519277' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi160-75741-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:15.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:15 smithi160 ceph-mon[30137]: osdmap e113: 8 total, 8 up, 8 in 2024-09-16T08:48:15.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:48:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:48:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:48:15.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:15 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:15.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:15 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1773519277' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi160-75741-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:15.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:15 smithi160 ceph-mon[40028]: osdmap e113: 8 total, 8 up, 8 in 2024-09-16T08:48:16.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:16.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:16 smithi169 ceph-mon[33465]: pgmap v117: 428 pgs: 64 unknown, 40 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-09-16T08:48:16.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:16.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:48:16.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:48:16.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:16 smithi169 ceph-mon[33465]: osdmap e114: 8 total, 8 up, 8 in 2024-09-16T08:48:16.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-13"} : dispatch 2024-09-16T08:48:16.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/303401962' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi160-75757-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:16.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:16.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:16 smithi160 ceph-mon[30137]: pgmap v117: 428 pgs: 64 unknown, 40 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-09-16T08:48:16.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:48:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:48:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:16 smithi160 ceph-mon[30137]: osdmap e114: 8 total, 8 up, 8 in 2024-09-16T08:48:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-13"} : dispatch 2024-09-16T08:48:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/303401962' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi160-75757-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:16.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:16.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:16 smithi160 ceph-mon[40028]: pgmap v117: 428 pgs: 64 unknown, 40 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-09-16T08:48:16.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:16.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:48:16.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:48:16.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:16 smithi160 ceph-mon[40028]: osdmap e114: 8 total, 8 up, 8 in 2024-09-16T08:48:16.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-13"} : dispatch 2024-09-16T08:48:16.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/303401962' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi160-75757-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:17.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:17.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:17 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:17.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:17 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:17.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-13"}]': finished 2024-09-16T08:48:17.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/303401962' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi160-75757-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:17.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:17 smithi160 ceph-mon[40028]: osdmap e115: 8 total, 8 up, 8 in 2024-09-16T08:48:17.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-13", "mode": "writeback"} : dispatch 2024-09-16T08:48:17.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/348420674' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi160-75741-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:17 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:17 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:17.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-13"}]': finished 2024-09-16T08:48:17.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/303401962' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi160-75757-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:17.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:17 smithi160 ceph-mon[30137]: osdmap e115: 8 total, 8 up, 8 in 2024-09-16T08:48:17.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-13", "mode": "writeback"} : dispatch 2024-09-16T08:48:17.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/348420674' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi160-75741-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:17.485 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:17.485 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:17 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:17.485 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:17 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:17.485 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-13"}]': finished 2024-09-16T08:48:17.485 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/303401962' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi160-75757-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:17.485 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:17 smithi169 ceph-mon[33465]: osdmap e115: 8 total, 8 up, 8 in 2024-09-16T08:48:17.485 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-13", "mode": "writeback"} : dispatch 2024-09-16T08:48:17.485 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/348420674' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi160-75741-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:18.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:18 smithi169 ceph-mon[33465]: pgmap v120: 396 pgs: 64 unknown, 8 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-09-16T08:48:18.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:18.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:18 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:48:18.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-13", "mode": "writeback"}]': finished 2024-09-16T08:48:18.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/348420674' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi160-75741-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:18.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:18 smithi169 ceph-mon[33465]: osdmap e116: 8 total, 8 up, 8 in 2024-09-16T08:48:18.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:18 smithi160 ceph-mon[40028]: pgmap v120: 396 pgs: 64 unknown, 8 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-09-16T08:48:18.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:18.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:18 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:48:18.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-13", "mode": "writeback"}]': finished 2024-09-16T08:48:18.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/348420674' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi160-75741-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:18.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:18 smithi160 ceph-mon[40028]: osdmap e116: 8 total, 8 up, 8 in 2024-09-16T08:48:18.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:18 smithi160 ceph-mon[30137]: pgmap v120: 396 pgs: 64 unknown, 8 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-09-16T08:48:18.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:18.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:18 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:48:18.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-13", "mode": "writeback"}]': finished 2024-09-16T08:48:18.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/348420674' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi160-75741-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:18.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:18 smithi160 ceph-mon[30137]: osdmap e116: 8 total, 8 up, 8 in 2024-09-16T08:48:19.173 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:19.173 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:19.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:48:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.0"} : dispatch 2024-09-16T08:48:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.1"} : dispatch 2024-09-16T08:48:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.2"} : dispatch 2024-09-16T08:48:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.3"} : dispatch 2024-09-16T08:48:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.4"} : dispatch 2024-09-16T08:48:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.5"} : dispatch 2024-09-16T08:48:19.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.6"} : dispatch 2024-09-16T08:48:19.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.7"} : dispatch 2024-09-16T08:48:19.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.8"} : dispatch 2024-09-16T08:48:19.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.9"} : dispatch 2024-09-16T08:48:19.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:48:19.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:48:19.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:19.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-16T08:48:19.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:19.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-16T08:48:19.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: osdmap e117: 8 total, 8 up, 8 in 2024-09-16T08:48:19.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3041075376' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi160-75757-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:19.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:19.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:19.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:19.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:19.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:48:19.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.0"} : dispatch 2024-09-16T08:48:19.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.1"} : dispatch 2024-09-16T08:48:19.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.2"} : dispatch 2024-09-16T08:48:19.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.3"} : dispatch 2024-09-16T08:48:19.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.4"} : dispatch 2024-09-16T08:48:19.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.5"} : dispatch 2024-09-16T08:48:19.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.6"} : dispatch 2024-09-16T08:48:19.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.7"} : dispatch 2024-09-16T08:48:19.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.8"} : dispatch 2024-09-16T08:48:19.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.9"} : dispatch 2024-09-16T08:48:19.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:48:19.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:48:19.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:19.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-16T08:48:19.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:19.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-16T08:48:19.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: osdmap e117: 8 total, 8 up, 8 in 2024-09-16T08:48:19.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3041075376' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi160-75757-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:19.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:48:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.0"} : dispatch 2024-09-16T08:48:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.1"} : dispatch 2024-09-16T08:48:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.2"} : dispatch 2024-09-16T08:48:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.3"} : dispatch 2024-09-16T08:48:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.4"} : dispatch 2024-09-16T08:48:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.5"} : dispatch 2024-09-16T08:48:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.6"} : dispatch 2024-09-16T08:48:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.7"} : dispatch 2024-09-16T08:48:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.8"} : dispatch 2024-09-16T08:48:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.9"} : dispatch 2024-09-16T08:48:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:48:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:48:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:48:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-16T08:48:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-16T08:48:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: osdmap e117: 8 total, 8 up, 8 in 2024-09-16T08:48:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3041075376' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi160-75757-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:20.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.0"}]: dispatch 2024-09-16T08:48:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.1"}]: dispatch 2024-09-16T08:48:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.2"}]: dispatch 2024-09-16T08:48:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.3"}]: dispatch 2024-09-16T08:48:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.4"}]: dispatch 2024-09-16T08:48:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.5"}]: dispatch 2024-09-16T08:48:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.6"}]: dispatch 2024-09-16T08:48:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.7"}]: dispatch 2024-09-16T08:48:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.8"}]: dispatch 2024-09-16T08:48:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.9"}]: dispatch 2024-09-16T08:48:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: 171.8 scrub starts 2024-09-16T08:48:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: 171.8 scrub ok 2024-09-16T08:48:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: pgmap v123: 396 pgs: 396 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 312 KiB/s wr, 129 op/s 2024-09-16T08:48:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: 171.4 scrub starts 2024-09-16T08:48:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: 171.4 scrub ok 2024-09-16T08:48:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: 171.6 scrub starts 2024-09-16T08:48:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: 171.6 scrub ok 2024-09-16T08:48:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3041075376' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi160-75757-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: osdmap e118: 8 total, 8 up, 8 in 2024-09-16T08:48:20.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/928690905' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi160-75741-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:20.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[30137]: from='client.56723 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi160-75741-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.0"}]: dispatch 2024-09-16T08:48:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.1"}]: dispatch 2024-09-16T08:48:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.2"}]: dispatch 2024-09-16T08:48:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.3"}]: dispatch 2024-09-16T08:48:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.4"}]: dispatch 2024-09-16T08:48:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.5"}]: dispatch 2024-09-16T08:48:20.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.6"}]: dispatch 2024-09-16T08:48:20.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.7"}]: dispatch 2024-09-16T08:48:20.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.8"}]: dispatch 2024-09-16T08:48:20.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.9"}]: dispatch 2024-09-16T08:48:20.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: 171.8 scrub starts 2024-09-16T08:48:20.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: 171.8 scrub ok 2024-09-16T08:48:20.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: pgmap v123: 396 pgs: 396 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 312 KiB/s wr, 129 op/s 2024-09-16T08:48:20.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:20.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: 171.4 scrub starts 2024-09-16T08:48:20.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: 171.4 scrub ok 2024-09-16T08:48:20.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: 171.6 scrub starts 2024-09-16T08:48:20.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: 171.6 scrub ok 2024-09-16T08:48:20.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3041075376' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi160-75757-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:20.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: osdmap e118: 8 total, 8 up, 8 in 2024-09-16T08:48:20.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/928690905' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi160-75741-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:20.452 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:20 smithi160 ceph-mon[40028]: from='client.56723 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi160-75741-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:20.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.0"}]: dispatch 2024-09-16T08:48:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.1"}]: dispatch 2024-09-16T08:48:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.2"}]: dispatch 2024-09-16T08:48:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.3"}]: dispatch 2024-09-16T08:48:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.4"}]: dispatch 2024-09-16T08:48:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.5"}]: dispatch 2024-09-16T08:48:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.6"}]: dispatch 2024-09-16T08:48:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.7"}]: dispatch 2024-09-16T08:48:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.8"}]: dispatch 2024-09-16T08:48:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.9"}]: dispatch 2024-09-16T08:48:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: 171.8 scrub starts 2024-09-16T08:48:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: 171.8 scrub ok 2024-09-16T08:48:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: pgmap v123: 396 pgs: 396 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 312 KiB/s wr, 129 op/s 2024-09-16T08:48:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: 171.4 scrub starts 2024-09-16T08:48:20.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: 171.4 scrub ok 2024-09-16T08:48:20.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: 171.6 scrub starts 2024-09-16T08:48:20.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: 171.6 scrub ok 2024-09-16T08:48:20.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3041075376' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi160-75757-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:20.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: osdmap e118: 8 total, 8 up, 8 in 2024-09-16T08:48:20.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/928690905' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi160-75741-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:20.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:20 smithi169 ceph-mon[33465]: from='client.56723 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi160-75741-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:21.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[30137]: 171.2 scrub starts 2024-09-16T08:48:21.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[30137]: 171.2 scrub ok 2024-09-16T08:48:21.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[30137]: 171.0 scrub starts 2024-09-16T08:48:21.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[30137]: 171.0 scrub ok 2024-09-16T08:48:21.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[30137]: 171.9 scrub starts 2024-09-16T08:48:21.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[30137]: 171.9 scrub ok 2024-09-16T08:48:21.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:21.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:21.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"} : dispatch 2024-09-16T08:48:21.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[30137]: from='client.24730 ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"} : dispatch 2024-09-16T08:48:21.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[30137]: 171.5 scrub starts 2024-09-16T08:48:21.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[30137]: 171.5 scrub ok 2024-09-16T08:48:21.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[30137]: from='client.56723 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi160-75741-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:21.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[30137]: from='client.24730 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"}]': finished 2024-09-16T08:48:21.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[30137]: osdmap e119: 8 total, 8 up, 8 in 2024-09-16T08:48:21.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:21.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[40028]: 171.2 scrub starts 2024-09-16T08:48:21.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[40028]: 171.2 scrub ok 2024-09-16T08:48:21.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[40028]: 171.0 scrub starts 2024-09-16T08:48:21.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[40028]: 171.0 scrub ok 2024-09-16T08:48:21.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[40028]: 171.9 scrub starts 2024-09-16T08:48:21.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[40028]: 171.9 scrub ok 2024-09-16T08:48:21.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:21.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:21.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"} : dispatch 2024-09-16T08:48:21.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[40028]: from='client.24730 ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"} : dispatch 2024-09-16T08:48:21.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[40028]: 171.5 scrub starts 2024-09-16T08:48:21.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[40028]: 171.5 scrub ok 2024-09-16T08:48:21.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[40028]: from='client.56723 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi160-75741-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:21.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[40028]: from='client.24730 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"}]': finished 2024-09-16T08:48:21.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[40028]: osdmap e119: 8 total, 8 up, 8 in 2024-09-16T08:48:21.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:21 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:21.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:21 smithi169 ceph-mon[33465]: 171.2 scrub starts 2024-09-16T08:48:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:21 smithi169 ceph-mon[33465]: 171.2 scrub ok 2024-09-16T08:48:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:21 smithi169 ceph-mon[33465]: 171.0 scrub starts 2024-09-16T08:48:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:21 smithi169 ceph-mon[33465]: 171.0 scrub ok 2024-09-16T08:48:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:21 smithi169 ceph-mon[33465]: 171.9 scrub starts 2024-09-16T08:48:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:21 smithi169 ceph-mon[33465]: 171.9 scrub ok 2024-09-16T08:48:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:21 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:21 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:21 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"} : dispatch 2024-09-16T08:48:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:21 smithi169 ceph-mon[33465]: from='client.24730 ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"} : dispatch 2024-09-16T08:48:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:21 smithi169 ceph-mon[33465]: 171.5 scrub starts 2024-09-16T08:48:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:21 smithi169 ceph-mon[33465]: 171.5 scrub ok 2024-09-16T08:48:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:21 smithi169 ceph-mon[33465]: from='client.56723 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi160-75741-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:21 smithi169 ceph-mon[33465]: from='client.24730 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"}]': finished 2024-09-16T08:48:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:21 smithi169 ceph-mon[33465]: osdmap e119: 8 total, 8 up, 8 in 2024-09-16T08:48:21.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:21 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:22.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[30137]: 171.3 scrub starts 2024-09-16T08:48:22.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[30137]: 171.3 scrub ok 2024-09-16T08:48:22.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[30137]: 171.1 scrub starts 2024-09-16T08:48:22.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[30137]: 171.1 scrub ok 2024-09-16T08:48:22.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[30137]: pgmap v126: 428 pgs: 96 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 56 KiB/s wr, 127 op/s 2024-09-16T08:48:22.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:22.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[30137]: osdmap e120: 8 total, 8 up, 8 in 2024-09-16T08:48:22.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4202797300' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi160-75757-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:22.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[30137]: from='client.56729 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi160-75757-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:22.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:22.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[30137]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:22.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[40028]: 171.3 scrub starts 2024-09-16T08:48:22.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[40028]: 171.3 scrub ok 2024-09-16T08:48:22.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[40028]: 171.1 scrub starts 2024-09-16T08:48:22.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[40028]: 171.1 scrub ok 2024-09-16T08:48:22.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[40028]: pgmap v126: 428 pgs: 96 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 56 KiB/s wr, 127 op/s 2024-09-16T08:48:22.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:22.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[40028]: osdmap e120: 8 total, 8 up, 8 in 2024-09-16T08:48:22.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4202797300' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi160-75757-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:22.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[40028]: from='client.56729 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi160-75757-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:22.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:22.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:22 smithi160 ceph-mon[40028]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:22.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:22 smithi169 ceph-mon[33465]: 171.3 scrub starts 2024-09-16T08:48:22.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:22 smithi169 ceph-mon[33465]: 171.3 scrub ok 2024-09-16T08:48:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:22 smithi169 ceph-mon[33465]: 171.1 scrub starts 2024-09-16T08:48:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:22 smithi169 ceph-mon[33465]: 171.1 scrub ok 2024-09-16T08:48:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:22 smithi169 ceph-mon[33465]: pgmap v126: 428 pgs: 96 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 56 KiB/s wr, 127 op/s 2024-09-16T08:48:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:22 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:22 smithi169 ceph-mon[33465]: osdmap e120: 8 total, 8 up, 8 in 2024-09-16T08:48:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:22 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4202797300' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi160-75757-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:22 smithi169 ceph-mon[33465]: from='client.56729 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi160-75757-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:22 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:22 smithi169 ceph-mon[33465]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:23.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[30137]: 171.7 scrub starts 2024-09-16T08:48:23.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[30137]: 171.7 scrub ok 2024-09-16T08:48:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[30137]: from='client.56729 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi160-75757-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[30137]: from='client.56681 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"}]': finished 2024-09-16T08:48:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[30137]: osdmap e121: 8 total, 8 up, 8 in 2024-09-16T08:48:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1293092637' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi160-75741-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[30137]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[30137]: from='client.56735 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi160-75741-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:23.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[40028]: 171.7 scrub starts 2024-09-16T08:48:23.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[40028]: 171.7 scrub ok 2024-09-16T08:48:23.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:23.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[40028]: from='client.56729 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi160-75757-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:23.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[40028]: from='client.56681 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"}]': finished 2024-09-16T08:48:23.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[40028]: osdmap e121: 8 total, 8 up, 8 in 2024-09-16T08:48:23.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:23.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1293092637' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi160-75741-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:23.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[40028]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:23.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:23 smithi160 ceph-mon[40028]: from='client.56735 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi160-75741-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:23.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:23 smithi169 ceph-mon[33465]: 171.7 scrub starts 2024-09-16T08:48:23.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:23 smithi169 ceph-mon[33465]: 171.7 scrub ok 2024-09-16T08:48:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:23 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:23 smithi169 ceph-mon[33465]: from='client.56729 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi160-75757-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:23 smithi169 ceph-mon[33465]: from='client.56681 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"}]': finished 2024-09-16T08:48:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:23 smithi169 ceph-mon[33465]: osdmap e121: 8 total, 8 up, 8 in 2024-09-16T08:48:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:23 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3311442226' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:23 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1293092637' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi160-75741-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:23 smithi169 ceph-mon[33465]: from='client.56681 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"} : dispatch 2024-09-16T08:48:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:23 smithi169 ceph-mon[33465]: from='client.56735 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi160-75741-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:24.140 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: Running main() from gmock_main.cc 2024-09-16T08:48:24.140 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-09-16T08:48:24.140 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] Global test environment set-up. 2024-09-16T08:48:24.140 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-09-16T08:48:24.140 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-09-16T08:48:24.140 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (2025 ms) 2024-09-16T08:48:24.140 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-09-16T08:48:24.140 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (2010 ms) 2024-09-16T08:48:24.140 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-09-16T08:48:24.140 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2020 ms) 2024-09-16T08:48:24.141 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-09-16T08:48:24.141 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2012 ms) 2024-09-16T08:48:24.141 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (8067 ms total) 2024-09-16T08:48:24.141 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: 2024-09-16T08:48:24.141 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-09-16T08:48:24.141 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-09-16T08:48:24.141 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (4061 ms) 2024-09-16T08:48:24.141 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-09-16T08:48:24.141 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4118 ms) 2024-09-16T08:48:24.141 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-09-16T08:48:24.141 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (4919 ms) 2024-09-16T08:48:24.141 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (13098 ms total) 2024-09-16T08:48:24.141 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: 2024-09-16T08:48:24.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-09-16T08:48:24.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-09-16T08:48:24.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (2935 ms) 2024-09-16T08:48:24.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-09-16T08:48:24.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2045 ms) 2024-09-16T08:48:24.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-09-16T08:48:24.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (1981 ms) 2024-09-16T08:48:24.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-09-16T08:48:24.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (2016 ms) 2024-09-16T08:48:24.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (8977 ms total) 2024-09-16T08:48:24.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: 2024-09-16T08:48:24.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-09-16T08:48:24.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-09-16T08:48:24.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4022 ms) 2024-09-16T08:48:24.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-09-16T08:48:24.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4019 ms) 2024-09-16T08:48:24.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8041 ms total) 2024-09-16T08:48:24.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: 2024-09-16T08:48:24.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] Global test environment tear-down 2024-09-16T08:48:24.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (55521 ms total) 2024-09-16T08:48:24.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-09-16T08:48:24.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:24 smithi160 ceph-mon[30137]: pgmap v129: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:24 smithi160 ceph-mon[30137]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:24 smithi160 ceph-mon[30137]: from='client.56681 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"}]': finished 2024-09-16T08:48:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:24 smithi160 ceph-mon[30137]: from='client.56735 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi160-75741-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:24 smithi160 ceph-mon[30137]: osdmap e122: 8 total, 8 up, 8 in 2024-09-16T08:48:24.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:24 smithi160 ceph-mon[40028]: pgmap v129: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:24.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:24.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:24 smithi160 ceph-mon[40028]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:24.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:24 smithi160 ceph-mon[40028]: from='client.56681 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"}]': finished 2024-09-16T08:48:24.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:24 smithi160 ceph-mon[40028]: from='client.56735 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi160-75741-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:24.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:24 smithi160 ceph-mon[40028]: osdmap e122: 8 total, 8 up, 8 in 2024-09-16T08:48:24.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:24 smithi169 ceph-mon[33465]: pgmap v129: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:24 smithi169 ceph-mon[33465]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:24 smithi169 ceph-mon[33465]: from='client.56681 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi160-76059-15"}]': finished 2024-09-16T08:48:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:24 smithi169 ceph-mon[33465]: from='client.56735 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi160-75741-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:24 smithi169 ceph-mon[33465]: osdmap e122: 8 total, 8 up, 8 in 2024-09-16T08:48:25.099 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:48:24 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:48:25.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:48:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:48:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:48:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:25 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-16T08:48:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:25 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-16T08:48:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:25 smithi160 ceph-mon[30137]: osdmap e123: 8 total, 8 up, 8 in 2024-09-16T08:48:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3079931698' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi160-75757-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:25 smithi160 ceph-mon[30137]: from='client.47908 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi160-75757-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4015010264' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76133-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:25 smithi160 ceph-mon[30137]: from='client.56747 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76133-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:25.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:25 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-16T08:48:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:25 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-16T08:48:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:25 smithi160 ceph-mon[40028]: osdmap e123: 8 total, 8 up, 8 in 2024-09-16T08:48:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3079931698' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi160-75757-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:25 smithi160 ceph-mon[40028]: from='client.47908 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi160-75757-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4015010264' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76133-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:25 smithi160 ceph-mon[40028]: from='client.56747 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76133-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:25.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:25.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:25 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-16T08:48:25.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:25 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-16T08:48:25.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:25 smithi169 ceph-mon[33465]: osdmap e123: 8 total, 8 up, 8 in 2024-09-16T08:48:25.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3079931698' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi160-75757-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:25.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:25 smithi169 ceph-mon[33465]: from='client.47908 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi160-75757-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:25.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4015010264' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76133-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:25.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:25 smithi169 ceph-mon[33465]: from='client.56747 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76133-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:26.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:26.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:26 smithi169 ceph-mon[33465]: pgmap v132: 356 pgs: 32 creating+peering, 32 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 31 B/s, 0 objects/s recovering 2024-09-16T08:48:26.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:26.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:26 smithi169 ceph-mon[33465]: from='client.47908 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi160-75757-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:26 smithi169 ceph-mon[33465]: from='client.56747 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76133-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:26 smithi169 ceph-mon[33465]: osdmap e124: 8 total, 8 up, 8 in 2024-09-16T08:48:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3000908470' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi160-75741-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:26.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:26.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:26 smithi160 ceph-mon[30137]: pgmap v132: 356 pgs: 32 creating+peering, 32 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 31 B/s, 0 objects/s recovering 2024-09-16T08:48:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:26 smithi160 ceph-mon[30137]: from='client.47908 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi160-75757-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:26 smithi160 ceph-mon[30137]: from='client.56747 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76133-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:26 smithi160 ceph-mon[30137]: osdmap e124: 8 total, 8 up, 8 in 2024-09-16T08:48:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3000908470' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi160-75741-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:26 smithi160 ceph-mon[40028]: pgmap v132: 356 pgs: 32 creating+peering, 32 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 31 B/s, 0 objects/s recovering 2024-09-16T08:48:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:26 smithi160 ceph-mon[40028]: from='client.47908 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi160-75757-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:26 smithi160 ceph-mon[40028]: from='client.56747 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76133-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:26 smithi160 ceph-mon[40028]: osdmap e124: 8 total, 8 up, 8 in 2024-09-16T08:48:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3000908470' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi160-75741-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:27.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3000908470' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi160-75741-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:27 smithi169 ceph-mon[33465]: osdmap e125: 8 total, 8 up, 8 in 2024-09-16T08:48:27.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3000908470' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi160-75741-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:27 smithi160 ceph-mon[30137]: osdmap e125: 8 total, 8 up, 8 in 2024-09-16T08:48:27.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:27.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3000908470' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi160-75741-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:27.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:27 smithi160 ceph-mon[40028]: osdmap e125: 8 total, 8 up, 8 in 2024-09-16T08:48:28.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:28 smithi169 ceph-mon[33465]: pgmap v135: 420 pgs: 128 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 0 B/s wr, 0 op/s; 31 B/s, 0 objects/s recovering 2024-09-16T08:48:28.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:28 smithi169 ceph-mon[33465]: osdmap e126: 8 total, 8 up, 8 in 2024-09-16T08:48:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4044347655' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi160-75757-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:28 smithi169 ceph-mon[33465]: from='client.47932 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi160-75757-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:28.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:28 smithi160 ceph-mon[30137]: pgmap v135: 420 pgs: 128 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 0 B/s wr, 0 op/s; 31 B/s, 0 objects/s recovering 2024-09-16T08:48:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:28 smithi160 ceph-mon[30137]: osdmap e126: 8 total, 8 up, 8 in 2024-09-16T08:48:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4044347655' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi160-75757-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:28 smithi160 ceph-mon[30137]: from='client.47932 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi160-75757-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:28 smithi160 ceph-mon[40028]: pgmap v135: 420 pgs: 128 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 0 B/s wr, 0 op/s; 31 B/s, 0 objects/s recovering 2024-09-16T08:48:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:28 smithi160 ceph-mon[40028]: osdmap e126: 8 total, 8 up, 8 in 2024-09-16T08:48:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4044347655' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi160-75757-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:28.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:28 smithi160 ceph-mon[40028]: from='client.47932 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi160-75757-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:29 smithi160 ceph-mon[30137]: from='client.47932 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi160-75757-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:29 smithi160 ceph-mon[30137]: osdmap e127: 8 total, 8 up, 8 in 2024-09-16T08:48:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/181679353' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi160-75741-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:29 smithi160 ceph-mon[30137]: from='client.47944 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi160-75741-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:29.600 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi160-76133-16"} : dispatch 2024-09-16T08:48:29.600 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi160-76133-16"} : dispatch 2024-09-16T08:48:29.600 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi160-76133-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:29 smithi160 ceph-mon[40028]: from='client.47932 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi160-75757-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:29 smithi160 ceph-mon[40028]: osdmap e127: 8 total, 8 up, 8 in 2024-09-16T08:48:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/181679353' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi160-75741-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:29 smithi160 ceph-mon[40028]: from='client.47944 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi160-75741-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi160-76133-16"} : dispatch 2024-09-16T08:48:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi160-76133-16"} : dispatch 2024-09-16T08:48:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi160-76133-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:29 smithi169 ceph-mon[33465]: from='client.47932 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi160-75757-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:29 smithi169 ceph-mon[33465]: osdmap e127: 8 total, 8 up, 8 in 2024-09-16T08:48:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/181679353' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi160-75741-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:29 smithi169 ceph-mon[33465]: from='client.47944 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi160-75741-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi160-76133-16"} : dispatch 2024-09-16T08:48:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi160-76133-16"} : dispatch 2024-09-16T08:48:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi160-76133-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:30.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:30 smithi169 ceph-mon[33465]: pgmap v138: 387 pgs: 32 creating+peering, 355 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:48:30.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:30.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:30 smithi169 ceph-mon[33465]: from='client.47944 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi160-75741-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi160-76133-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:30 smithi169 ceph-mon[33465]: osdmap e128: 8 total, 8 up, 8 in 2024-09-16T08:48:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi160-76133-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi160-76133-16"} : dispatch 2024-09-16T08:48:30.611 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:30 smithi160 ceph-mon[30137]: pgmap v138: 387 pgs: 32 creating+peering, 355 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:48:30.611 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:30.612 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:30 smithi160 ceph-mon[30137]: from='client.47944 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi160-75741-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:30.612 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi160-76133-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:30.612 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:30 smithi160 ceph-mon[30137]: osdmap e128: 8 total, 8 up, 8 in 2024-09-16T08:48:30.612 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi160-76133-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi160-76133-16"} : dispatch 2024-09-16T08:48:30.612 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:30 smithi160 ceph-mon[40028]: pgmap v138: 387 pgs: 32 creating+peering, 355 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:48:30.612 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:30.612 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:30 smithi160 ceph-mon[40028]: from='client.47944 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi160-75741-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:30.612 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi160-76133-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:30.612 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:30 smithi160 ceph-mon[40028]: osdmap e128: 8 total, 8 up, 8 in 2024-09-16T08:48:30.612 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi160-76133-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi160-76133-16"} : dispatch 2024-09-16T08:48:31.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:31.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:31 smithi169 ceph-mon[33465]: osdmap e129: 8 total, 8 up, 8 in 2024-09-16T08:48:31.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4244655354' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi160-75757-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:31.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:31.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:31 smithi160 ceph-mon[40028]: osdmap e129: 8 total, 8 up, 8 in 2024-09-16T08:48:31.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4244655354' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi160-75757-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:31 smithi160 ceph-mon[30137]: osdmap e129: 8 total, 8 up, 8 in 2024-09-16T08:48:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4244655354' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi160-75757-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:32.501 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:32 smithi169 ceph-mon[33465]: pgmap v141: 355 pgs: 32 unknown, 323 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:48:32.501 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:32 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:32.501 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:32.501 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi160-76133-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi160-76133-16"}]': finished 2024-09-16T08:48:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4244655354' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi160-75757-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:32 smithi169 ceph-mon[33465]: osdmap e130: 8 total, 8 up, 8 in 2024-09-16T08:48:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2663019938' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi160-75741-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:32 smithi169 ceph-mon[33465]: from='client.56804 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi160-75741-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:32.502 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:32.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[30137]: pgmap v141: 355 pgs: 32 unknown, 323 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:48:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi160-76133-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi160-76133-16"}]': finished 2024-09-16T08:48:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4244655354' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi160-75757-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[30137]: osdmap e130: 8 total, 8 up, 8 in 2024-09-16T08:48:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2663019938' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi160-75741-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[30137]: from='client.56804 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi160-75741-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[40028]: pgmap v141: 355 pgs: 32 unknown, 323 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:48:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi160-76133-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi160-76133-16"}]': finished 2024-09-16T08:48:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4244655354' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi160-75757-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[40028]: osdmap e130: 8 total, 8 up, 8 in 2024-09-16T08:48:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2663019938' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi160-75741-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[40028]: from='client.56804 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi160-75741-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:34.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[30137]: pgmap v144: 395 pgs: 72 unknown, 323 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[30137]: from='client.56804 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi160-75741-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[30137]: osdmap e131: 8 total, 8 up, 8 in 2024-09-16T08:48:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:48:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-16T08:48:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[30137]: from='client.24745 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[30137]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[30137]: osdmap e132: 8 total, 8 up, 8 in 2024-09-16T08:48:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app2"} : dispatch 2024-09-16T08:48:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[30137]: from='client.24745 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4076213315' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi160-75757-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[30137]: from='client.56810 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi160-75757-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[40028]: pgmap v144: 395 pgs: 72 unknown, 323 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:34.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[40028]: from='client.56804 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi160-75741-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:34.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[40028]: osdmap e131: 8 total, 8 up, 8 in 2024-09-16T08:48:34.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:34.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:48:34.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-16T08:48:34.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[40028]: from='client.24745 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[40028]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:34.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[40028]: osdmap e132: 8 total, 8 up, 8 in 2024-09-16T08:48:34.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app2"} : dispatch 2024-09-16T08:48:34.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[40028]: from='client.24745 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4076213315' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi160-75757-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-mon[40028]: from='client.56810 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi160-75757-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:34 smithi169 ceph-mon[33465]: pgmap v144: 395 pgs: 72 unknown, 323 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:34 smithi169 ceph-mon[33465]: from='client.56804 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi160-75741-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:34 smithi169 ceph-mon[33465]: osdmap e131: 8 total, 8 up, 8 in 2024-09-16T08:48:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:48:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-16T08:48:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:34 smithi169 ceph-mon[33465]: from='client.24745 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:34 smithi169 ceph-mon[33465]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:34 smithi169 ceph-mon[33465]: osdmap e132: 8 total, 8 up, 8 in 2024-09-16T08:48:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app2"} : dispatch 2024-09-16T08:48:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:34 smithi169 ceph-mon[33465]: from='client.24745 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4076213315' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi160-75757-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:34 smithi169 ceph-mon[33465]: from='client.56810 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi160-75757-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:35.158 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:48:34 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:48:35.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:48:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:48:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:48:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:35 smithi160 ceph-mon[40028]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:35 smithi160 ceph-mon[40028]: from='client.56810 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi160-75757-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:35 smithi160 ceph-mon[40028]: osdmap e133: 8 total, 8 up, 8 in 2024-09-16T08:48:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2552207313' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi160-75741-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"dne","key":"key1","value":"value1"} : dispatch 2024-09-16T08:48:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-16T08:48:35.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:35 smithi160 ceph-mon[40028]: from='client.24745 ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-16T08:48:35.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:35.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:35 smithi160 ceph-mon[30137]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:35.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:35 smithi160 ceph-mon[30137]: from='client.56810 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi160-75757-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:35.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:35 smithi160 ceph-mon[30137]: osdmap e133: 8 total, 8 up, 8 in 2024-09-16T08:48:35.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2552207313' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi160-75741-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:35.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"dne","key":"key1","value":"value1"} : dispatch 2024-09-16T08:48:35.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-16T08:48:35.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:35 smithi160 ceph-mon[30137]: from='client.24745 ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-16T08:48:35.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:35 smithi169 ceph-mon[33465]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi160-75993-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:35 smithi169 ceph-mon[33465]: from='client.56810 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi160-75757-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:35 smithi169 ceph-mon[33465]: osdmap e133: 8 total, 8 up, 8 in 2024-09-16T08:48:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2552207313' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi160-75741-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"dne","key":"key1","value":"value1"} : dispatch 2024-09-16T08:48:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-16T08:48:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:35 smithi169 ceph-mon[33465]: from='client.24745 ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-16T08:48:36.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:36 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:36.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:36 smithi160 ceph-mon[30137]: pgmap v147: 363 pgs: 32 unknown, 331 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:48:36.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:36.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2552207313' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi160-75741-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:36.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:36 smithi160 ceph-mon[30137]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-16T08:48:36.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:36 smithi160 ceph-mon[30137]: osdmap e134: 8 total, 8 up, 8 in 2024-09-16T08:48:36.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-16T08:48:36.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:36 smithi160 ceph-mon[30137]: from='client.24745 ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-16T08:48:36.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:36 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:36.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:36 smithi160 ceph-mon[40028]: pgmap v147: 363 pgs: 32 unknown, 331 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:48:36.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:36.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2552207313' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi160-75741-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:36.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:36 smithi160 ceph-mon[40028]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-16T08:48:36.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:36 smithi160 ceph-mon[40028]: osdmap e134: 8 total, 8 up, 8 in 2024-09-16T08:48:36.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-16T08:48:36.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:36 smithi160 ceph-mon[40028]: from='client.24745 ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-16T08:48:36.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:36 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:36.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:36 smithi169 ceph-mon[33465]: pgmap v147: 363 pgs: 32 unknown, 331 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:48:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2552207313' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi160-75741-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:36 smithi169 ceph-mon[33465]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-16T08:48:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:36 smithi169 ceph-mon[33465]: osdmap e134: 8 total, 8 up, 8 in 2024-09-16T08:48:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-16T08:48:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:36 smithi169 ceph-mon[33465]: from='client.24745 ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-16T08:48:37.133 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: Running main() from gmock_main.cc 2024-09-16T08:48:37.133 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-09-16T08:48:37.133 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-09-16T08:48:37.133 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-09-16T08:48:37.133 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: seed 76133 2024-09-16T08:48:37.133 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-09-16T08:48:37.133 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (2024 ms) 2024-09-16T08:48:37.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-09-16T08:48:37.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (2012 ms) 2024-09-16T08:48:37.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-09-16T08:48:37.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (2023 ms) 2024-09-16T08:48:37.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-09-16T08:48:37.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2022 ms) 2024-09-16T08:48:37.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-09-16T08:48:37.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (3884 ms) 2024-09-16T08:48:37.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (11966 ms total) 2024-09-16T08:48:37.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: 2024-09-16T08:48:37.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-09-16T08:48:37.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-09-16T08:48:37.134 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4182 ms) 2024-09-16T08:48:37.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-09-16T08:48:37.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (3872 ms) 2024-09-16T08:48:37.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-09-16T08:48:37.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (5967 ms) 2024-09-16T08:48:37.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-09-16T08:48:37.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (4011 ms) 2024-09-16T08:48:37.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-09-16T08:48:37.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2104 ms) 2024-09-16T08:48:37.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-09-16T08:48:37.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4917-g8293d73f/rpm/el9/BUILD/ceph-19.3.0-4917-g8293d73f/src/test/librados/snapshots_cxx.cc:460: Skipped 2024-09-16T08:48:37.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: 2024-09-16T08:48:37.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-09-16T08:48:37.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-09-16T08:48:37.135 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi160-76133-7 2024-09-16T08:48:37.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: waiting for snaps to purge 2024-09-16T08:48:37.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17913 ms) 2024-09-16T08:48:37.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (38049 ms total) 2024-09-16T08:48:37.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: 2024-09-16T08:48:37.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-09-16T08:48:37.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-09-16T08:48:37.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (10 ms) 2024-09-16T08:48:37.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-09-16T08:48:37.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5041 ms) 2024-09-16T08:48:37.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5051 ms total) 2024-09-16T08:48:37.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: 2024-09-16T08:48:37.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-09-16T08:48:37.136 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-09-16T08:48:37.137 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (2971 ms) 2024-09-16T08:48:37.137 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-09-16T08:48:37.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:37.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:37 smithi160 ceph-mon[30137]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-16T08:48:37.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:37 smithi160 ceph-mon[30137]: osdmap e135: 8 total, 8 up, 8 in 2024-09-16T08:48:37.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1"} : dispatch 2024-09-16T08:48:37.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2352032071' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi160-75757-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:37.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:37 smithi160 ceph-mon[30137]: from='client.56819 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi160-75757-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:37.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:37 smithi160 ceph-mon[30137]: from='client.24745 ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1"} : dispatch 2024-09-16T08:48:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:37 smithi160 ceph-mon[40028]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-16T08:48:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:37 smithi160 ceph-mon[40028]: osdmap e135: 8 total, 8 up, 8 in 2024-09-16T08:48:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1"} : dispatch 2024-09-16T08:48:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2352032071' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi160-75757-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:37.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:37 smithi160 ceph-mon[40028]: from='client.56819 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi160-75757-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:37.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:37 smithi160 ceph-mon[40028]: from='client.24745 ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1"} : dispatch 2024-09-16T08:48:37.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:37 smithi169 ceph-mon[33465]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-16T08:48:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:37 smithi169 ceph-mon[33465]: osdmap e135: 8 total, 8 up, 8 in 2024-09-16T08:48:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/569026314' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1"} : dispatch 2024-09-16T08:48:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2352032071' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi160-75757-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:37 smithi169 ceph-mon[33465]: from='client.56819 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi160-75757-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:37 smithi169 ceph-mon[33465]: from='client.24745 ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1"} : dispatch 2024-09-16T08:48:38.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:38 smithi160 ceph-mon[40028]: pgmap v150: 363 pgs: 32 unknown, 331 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:48:38.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:38 smithi160 ceph-mon[40028]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:38.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:38.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:38 smithi160 ceph-mon[40028]: from='client.56819 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi160-75757-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:38.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:38 smithi160 ceph-mon[40028]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1"}]': finished 2024-09-16T08:48:38.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:38 smithi160 ceph-mon[40028]: osdmap e136: 8 total, 8 up, 8 in 2024-09-16T08:48:38.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2885418596' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClass_smithi160-75741-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:38.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:38 smithi160 ceph-mon[30137]: pgmap v150: 363 pgs: 32 unknown, 331 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:48:38.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:38 smithi160 ceph-mon[30137]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:38 smithi160 ceph-mon[30137]: from='client.56819 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi160-75757-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:38 smithi160 ceph-mon[30137]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1"}]': finished 2024-09-16T08:48:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:38 smithi160 ceph-mon[30137]: osdmap e136: 8 total, 8 up, 8 in 2024-09-16T08:48:38.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2885418596' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClass_smithi160-75741-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:38.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:38 smithi169 ceph-mon[33465]: pgmap v150: 363 pgs: 32 unknown, 331 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:48:38.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:38 smithi169 ceph-mon[33465]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:38 smithi169 ceph-mon[33465]: from='client.56819 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi160-75757-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:38 smithi169 ceph-mon[33465]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi160-75993-1","app":"app1","key":"key1"}]': finished 2024-09-16T08:48:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:38 smithi169 ceph-mon[33465]: osdmap e136: 8 total, 8 up, 8 in 2024-09-16T08:48:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2885418596' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClass_smithi160-75741-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:39.147 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRados api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-09-16T08:48:39.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-09-16T08:48:39.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-09-16T08:48:39.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-09-16T08:48:39.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-09-16T08:48:39.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-16T08:48:39.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: 2024-09-16T08:48:39.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-09-16T08:48:39.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: seed 75993 2024-09-16T08:48:39.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-09-16T08:48:39.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (2 ms) 2024-09-16T08:48:39.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-09-16T08:48:39.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (413 ms) 2024-09-16T08:48:39.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-09-16T08:48:39.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (16 ms) 2024-09-16T08:48:39.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-09-16T08:48:39.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (8 ms) 2024-09-16T08:48:39.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-09-16T08:48:39.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (14 ms) 2024-09-16T08:48:39.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-09-16T08:48:39.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (3 ms) 2024-09-16T08:48:39.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-09-16T08:48:39.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (4 ms) 2024-09-16T08:48:39.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-09-16T08:48:39.150 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (5 ms) 2024-09-16T08:48:39.150 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-09-16T08:48:39.150 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-09-16T08:48:39.150 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-09-16T08:48:39.150 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (14 ms) 2024-09-16T08:48:39.150 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-09-16T08:48:39.150 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (2 ms) 2024-09-16T08:48:39.150 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-09-16T08:48:39.150 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (6 ms) 2024-09-16T08:48:39.150 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-09-16T08:48:39.150 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (10 ms) 2024-09-16T08:48:39.150 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-09-16T08:48:39.150 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: osd_max_attr_size = 0 2024-09-16T08:48:39.150 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-09-16T08:48:39.151 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1716 ms) 2024-09-16T08:48:39.151 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-09-16T08:48:39.151 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (385 ms) 2024-09-16T08:48:39.151 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-09-16T08:48:39.151 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-09-16T08:48:39.151 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: done waiting, doing copies 2024-09-16T08:48:39.151 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: waiting for final deep scrubs... 2024-09-16T08:48:39.151 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: done waiting 2024-09-16T08:48:39.151 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60844 ms) 2024-09-16T08:48:39.151 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-09-16T08:48:39.151 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (5 ms) 2024-09-16T08:48:39.151 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-09-16T08:48:39.151 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (3 ms) 2024-09-16T08:48:39.151 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-09-16T08:48:39.152 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4540 ms) 2024-09-16T08:48:39.152 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-09-16T08:48:39.152 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-09-16T08:48:39.152 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-09-16T08:48:39.152 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-09-16T08:48:39.152 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-09-16T08:48:39.152 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-09-16T08:48:39.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:39.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2885418596' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi160-75741-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:39.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:39 smithi160 ceph-mon[30137]: osdmap e137: 8 total, 8 up, 8 in 2024-09-16T08:48:39.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:39.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2885418596' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi160-75741-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:39.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:39 smithi160 ceph-mon[40028]: osdmap e137: 8 total, 8 up, 8 in 2024-09-16T08:48:39.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:39.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2885418596' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi160-75741-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:39.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:39 smithi169 ceph-mon[33465]: osdmap e137: 8 total, 8 up, 8 in 2024-09-16T08:48:40.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[30137]: pgmap v153: 395 pgs: 32 creating+peering, 363 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:48:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-75993-24"} : dispatch 2024-09-16T08:48:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-75993-24"} : dispatch 2024-09-16T08:48:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-75993-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-75993-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[30137]: osdmap e138: 8 total, 8 up, 8 in 2024-09-16T08:48:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-75993-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-75993-24"} : dispatch 2024-09-16T08:48:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2779147427' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi160-75757-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:40.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[40028]: pgmap v153: 395 pgs: 32 creating+peering, 363 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:48:40.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-75993-24"} : dispatch 2024-09-16T08:48:40.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-75993-24"} : dispatch 2024-09-16T08:48:40.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-75993-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:40.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:40.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-75993-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:40.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[40028]: osdmap e138: 8 total, 8 up, 8 in 2024-09-16T08:48:40.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-75993-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-75993-24"} : dispatch 2024-09-16T08:48:40.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2779147427' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi160-75757-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:40.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:40 smithi169 ceph-mon[33465]: pgmap v153: 395 pgs: 32 creating+peering, 363 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:48:40.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-75993-24"} : dispatch 2024-09-16T08:48:40.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-75993-24"} : dispatch 2024-09-16T08:48:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-75993-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-75993-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:40 smithi169 ceph-mon[33465]: osdmap e138: 8 total, 8 up, 8 in 2024-09-16T08:48:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-75993-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-75993-24"} : dispatch 2024-09-16T08:48:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2779147427' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi160-75757-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:41.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:41.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2779147427' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi160-75757-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:41.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:41 smithi160 ceph-mon[30137]: osdmap e139: 8 total, 8 up, 8 in 2024-09-16T08:48:41.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3675875799' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWrite_smithi160-75741-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:41.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:41 smithi160 ceph-mon[30137]: from='client.56837 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWrite_smithi160-75741-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:41.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:41.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2779147427' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi160-75757-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:41.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:41 smithi160 ceph-mon[40028]: osdmap e139: 8 total, 8 up, 8 in 2024-09-16T08:48:41.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3675875799' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWrite_smithi160-75741-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:41.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:41 smithi160 ceph-mon[40028]: from='client.56837 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWrite_smithi160-75741-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:41.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2779147427' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi160-75757-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:41 smithi169 ceph-mon[33465]: osdmap e139: 8 total, 8 up, 8 in 2024-09-16T08:48:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3675875799' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWrite_smithi160-75741-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:41 smithi169 ceph-mon[33465]: from='client.56837 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWrite_smithi160-75741-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:42.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:42 smithi160 ceph-mon[40028]: pgmap v156: 331 pgs: 32 unknown, 299 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:42.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:42.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi160-75993-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-75993-24"}]': finished 2024-09-16T08:48:42.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:42 smithi160 ceph-mon[40028]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi160-75741-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:42.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:42 smithi160 ceph-mon[40028]: osdmap e140: 8 total, 8 up, 8 in 2024-09-16T08:48:42.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi160-76133-16"} : dispatch 2024-09-16T08:48:42.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:42 smithi160 ceph-mon[30137]: pgmap v156: 331 pgs: 32 unknown, 299 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:42.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:42.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi160-75993-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-75993-24"}]': finished 2024-09-16T08:48:42.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:42 smithi160 ceph-mon[30137]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi160-75741-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:42.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:42 smithi160 ceph-mon[30137]: osdmap e140: 8 total, 8 up, 8 in 2024-09-16T08:48:42.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi160-76133-16"} : dispatch 2024-09-16T08:48:42.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:42 smithi169 ceph-mon[33465]: pgmap v156: 331 pgs: 32 unknown, 299 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi160-75993-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-75993-24"}]': finished 2024-09-16T08:48:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:42 smithi169 ceph-mon[33465]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi160-75741-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:42 smithi169 ceph-mon[33465]: osdmap e140: 8 total, 8 up, 8 in 2024-09-16T08:48:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi160-76133-16"} : dispatch 2024-09-16T08:48:43.153 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP snapshots: Running main() from gmock_main.cc 2024-09-16T08:48:43.153 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-09-16T08:48:43.153 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [----------] Global test environment set-up. 2024-09-16T08:48:43.153 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-09-16T08:48:43.153 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-09-16T08:48:43.153 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (4244 ms) 2024-09-16T08:48:43.153 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-09-16T08:48:43.153 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (5007 ms) 2024-09-16T08:48:43.153 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-09-16T08:48:43.153 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (3903 ms) 2024-09-16T08:48:43.154 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-09-16T08:48:43.154 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (5050 ms) 2024-09-16T08:48:43.154 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-09-16T08:48:43.154 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (6960 ms) 2024-09-16T08:48:43.154 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (25164 ms total) 2024-09-16T08:48:43.154 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: 2024-09-16T08:48:43.154 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-09-16T08:48:43.154 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-09-16T08:48:43.154 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (4994 ms) 2024-09-16T08:48:43.154 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-09-16T08:48:43.154 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (6079 ms) 2024-09-16T08:48:43.154 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-09-16T08:48:43.155 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (7924 ms) 2024-09-16T08:48:43.155 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-09-16T08:48:43.155 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (5994 ms) 2024-09-16T08:48:43.155 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-09-16T08:48:43.155 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4022 ms) 2024-09-16T08:48:43.155 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-09-16T08:48:43.155 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi160-77125-11. 2024-09-16T08:48:43.155 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: Waiting for snaps to purge. 2024-09-16T08:48:43.155 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (20007 ms) 2024-09-16T08:48:43.155 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (49020 ms total) 2024-09-16T08:48:43.155 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: 2024-09-16T08:48:43.155 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [----------] Global test environment tear-down 2024-09-16T08:48:43.156 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (74184 ms total) 2024-09-16T08:48:43.156 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ PASSED ] 11 tests. 2024-09-16T08:48:43.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi160-76133-16"}]': finished 2024-09-16T08:48:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:43 smithi169 ceph-mon[33465]: osdmap e141: 8 total, 8 up, 8 in 2024-09-16T08:48:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi160-76133-16"} : dispatch 2024-09-16T08:48:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2388289595' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi160-75757-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:43 smithi169 ceph-mon[33465]: from='client.56843 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi160-75757-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75993-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:43.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:43.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi160-76133-16"}]': finished 2024-09-16T08:48:43.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:43 smithi160 ceph-mon[40028]: osdmap e141: 8 total, 8 up, 8 in 2024-09-16T08:48:43.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi160-76133-16"} : dispatch 2024-09-16T08:48:43.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2388289595' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi160-75757-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:43.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:43 smithi160 ceph-mon[40028]: from='client.56843 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi160-75757-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:43.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75993-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:43.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:43.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi160-76133-16"}]': finished 2024-09-16T08:48:43.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:43 smithi160 ceph-mon[30137]: osdmap e141: 8 total, 8 up, 8 in 2024-09-16T08:48:43.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi160-76133-16"} : dispatch 2024-09-16T08:48:43.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2388289595' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi160-75757-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:43.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:43 smithi160 ceph-mon[30137]: from='client.56843 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi160-75757-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:43.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75993-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:44 smithi169 ceph-mon[33465]: pgmap v159: 331 pgs: 40 unknown, 291 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:44 smithi169 ceph-mon[33465]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi160-76133-16"}]': finished 2024-09-16T08:48:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:44 smithi169 ceph-mon[33465]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi160-75757-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75993-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:44 smithi169 ceph-mon[33465]: osdmap e142: 8 total, 8 up, 8 in 2024-09-16T08:48:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75993-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2663922317' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi160-75741-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:44 smithi169 ceph-mon[33465]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:44 smithi169 ceph-mon[33465]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:44 smithi169 ceph-mon[33465]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[30137]: pgmap v159: 331 pgs: 40 unknown, 291 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[30137]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi160-76133-16"}]': finished 2024-09-16T08:48:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[30137]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi160-75757-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75993-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[30137]: osdmap e142: 8 total, 8 up, 8 in 2024-09-16T08:48:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75993-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2663922317' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi160-75741-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[30137]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[30137]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[30137]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:44.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[40028]: pgmap v159: 331 pgs: 40 unknown, 291 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-16T08:48:44.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[40028]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:44.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:44.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3268796752' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi160-76133-16"}]': finished 2024-09-16T08:48:44.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[40028]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi160-75757-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:44.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75993-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[40028]: osdmap e142: 8 total, 8 up, 8 in 2024-09-16T08:48:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75993-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2663922317' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi160-75741-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[40028]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[40028]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-mon[40028]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:45.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:48:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:48:45.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:48:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:48:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:48:45.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:45.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75993-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:45.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2663922317' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi160-75741-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:45.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:45 smithi160 ceph-mon[40028]: from='client.48022 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:45.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:45.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:45 smithi160 ceph-mon[40028]: osdmap e143: 8 total, 8 up, 8 in 2024-09-16T08:48:45.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:45 smithi160 ceph-mon[40028]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:45.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:45.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75993-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:45.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2663922317' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi160-75741-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:45.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:45 smithi160 ceph-mon[30137]: from='client.48022 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:45.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:45.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:45 smithi160 ceph-mon[30137]: osdmap e143: 8 total, 8 up, 8 in 2024-09-16T08:48:45.451 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:45 smithi160 ceph-mon[30137]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:45.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:45.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75993-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:45.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2663922317' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi160-75741-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:45.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:45 smithi169 ceph-mon[33465]: from='client.48022 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:45.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:45.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:45 smithi169 ceph-mon[33465]: osdmap e143: 8 total, 8 up, 8 in 2024-09-16T08:48:45.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:45 smithi169 ceph-mon[33465]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:48:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:46 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:46 smithi169 ceph-mon[33465]: pgmap v162: 363 pgs: 32 unknown, 64 creating+peering, 267 active+clean; 592 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:48:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:46 smithi169 ceph-mon[33465]: osdmap e144: 8 total, 8 up, 8 in 2024-09-16T08:48:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1675395611' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi160-75757-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi160-75741-27"} : dispatch 2024-09-16T08:48:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi160-75741-27"} : dispatch 2024-09-16T08:48:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi160-75741-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:46.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:46 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:46 smithi160 ceph-mon[30137]: pgmap v162: 363 pgs: 32 unknown, 64 creating+peering, 267 active+clean; 592 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:48:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:46 smithi160 ceph-mon[30137]: osdmap e144: 8 total, 8 up, 8 in 2024-09-16T08:48:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1675395611' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi160-75757-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi160-75741-27"} : dispatch 2024-09-16T08:48:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi160-75741-27"} : dispatch 2024-09-16T08:48:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi160-75741-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:46 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:46 smithi160 ceph-mon[40028]: pgmap v162: 363 pgs: 32 unknown, 64 creating+peering, 267 active+clean; 592 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:48:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:46 smithi160 ceph-mon[40028]: osdmap e144: 8 total, 8 up, 8 in 2024-09-16T08:48:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1675395611' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi160-75757-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi160-75741-27"} : dispatch 2024-09-16T08:48:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi160-75741-27"} : dispatch 2024-09-16T08:48:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi160-75741-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:47.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:47 smithi169 ceph-mon[33465]: from='client.48022 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"}]': finished 2024-09-16T08:48:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1675395611' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi160-75757-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi160-75741-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:47 smithi169 ceph-mon[33465]: osdmap e145: 8 total, 8 up, 8 in 2024-09-16T08:48:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi160-75741-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi160-75741-27"} : dispatch 2024-09-16T08:48:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-75993-24"} : dispatch 2024-09-16T08:48:47.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:47.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:47 smithi160 ceph-mon[40028]: from='client.48022 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"}]': finished 2024-09-16T08:48:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1675395611' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi160-75757-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi160-75741-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:47 smithi160 ceph-mon[40028]: osdmap e145: 8 total, 8 up, 8 in 2024-09-16T08:48:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi160-75741-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi160-75741-27"} : dispatch 2024-09-16T08:48:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-75993-24"} : dispatch 2024-09-16T08:48:47.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:47.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:47 smithi160 ceph-mon[30137]: from='client.48022 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"}]': finished 2024-09-16T08:48:47.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1675395611' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi160-75757-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:47.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi160-75741-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:47.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:47 smithi160 ceph-mon[30137]: osdmap e145: 8 total, 8 up, 8 in 2024-09-16T08:48:47.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi160-75741-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi160-75741-27"} : dispatch 2024-09-16T08:48:47.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-75993-24"} : dispatch 2024-09-16T08:48:48.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:48 smithi169 ceph-mon[33465]: pgmap v165: 299 pgs: 32 unknown, 267 active+clean; 592 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:48:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-75993-24"}]': finished 2024-09-16T08:48:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:48 smithi169 ceph-mon[33465]: osdmap e146: 8 total, 8 up, 8 in 2024-09-16T08:48:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-75993-24"} : dispatch 2024-09-16T08:48:48.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:48 smithi160 ceph-mon[30137]: pgmap v165: 299 pgs: 32 unknown, 267 active+clean; 592 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:48:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-75993-24"}]': finished 2024-09-16T08:48:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:48 smithi160 ceph-mon[30137]: osdmap e146: 8 total, 8 up, 8 in 2024-09-16T08:48:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-75993-24"} : dispatch 2024-09-16T08:48:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:48 smithi160 ceph-mon[40028]: pgmap v165: 299 pgs: 32 unknown, 267 active+clean; 592 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:48:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-75993-24"}]': finished 2024-09-16T08:48:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:48 smithi160 ceph-mon[40028]: osdmap e146: 8 total, 8 up, 8 in 2024-09-16T08:48:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-75993-24"} : dispatch 2024-09-16T08:48:49.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:48:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pgp_num_actual", "val": "30"} : dispatch 2024-09-16T08:48:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:49 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi160-75741-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi160-75741-27"}]': finished 2024-09-16T08:48:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-75993-24"}]': finished 2024-09-16T08:48:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-16T08:48:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:49 smithi169 ceph-mon[33465]: osdmap e147: 8 total, 8 up, 8 in 2024-09-16T08:48:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/696419084' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi160-75757-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:49 smithi169 ceph-mon[33465]: from='client.56873 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi160-75757-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:49.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:48:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pgp_num_actual", "val": "30"} : dispatch 2024-09-16T08:48:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi160-75741-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi160-75741-27"}]': finished 2024-09-16T08:48:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-75993-24"}]': finished 2024-09-16T08:48:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-16T08:48:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[30137]: osdmap e147: 8 total, 8 up, 8 in 2024-09-16T08:48:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/696419084' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi160-75757-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[30137]: from='client.56873 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi160-75757-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:48:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pgp_num_actual", "val": "30"} : dispatch 2024-09-16T08:48:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi160-75741-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi160-75741-27"}]': finished 2024-09-16T08:48:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3691506874' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-75993-24"}]': finished 2024-09-16T08:48:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi160-75864-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-16T08:48:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[40028]: osdmap e147: 8 total, 8 up, 8 in 2024-09-16T08:48:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/696419084' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi160-75757-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:49 smithi160 ceph-mon[40028]: from='client.56873 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi160-75757-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:50 smithi169 ceph-mon[33465]: pgmap v168: 267 pgs: 267 active+clean; 600 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-09-16T08:48:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"} : dispatch 2024-09-16T08:48:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:50 smithi169 ceph-mon[33465]: from='client.24730 ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"} : dispatch 2024-09-16T08:48:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:50 smithi169 ceph-mon[33465]: from='client.56873 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi160-75757-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:50 smithi169 ceph-mon[33465]: from='client.24730 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"}]': finished 2024-09-16T08:48:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:50 smithi169 ceph-mon[33465]: osdmap e148: 8 total, 8 up, 8 in 2024-09-16T08:48:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2515142612' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi160-75993-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:50.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"} : dispatch 2024-09-16T08:48:50.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:50 smithi169 ceph-mon[33465]: from='client.56876 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi160-75993-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:50.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:50 smithi169 ceph-mon[33465]: from='client.24730 ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"} : dispatch 2024-09-16T08:48:50.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[30137]: pgmap v168: 267 pgs: 267 active+clean; 600 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-09-16T08:48:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"} : dispatch 2024-09-16T08:48:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[30137]: from='client.24730 ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"} : dispatch 2024-09-16T08:48:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[30137]: from='client.56873 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi160-75757-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[30137]: from='client.24730 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"}]': finished 2024-09-16T08:48:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[30137]: osdmap e148: 8 total, 8 up, 8 in 2024-09-16T08:48:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2515142612' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi160-75993-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"} : dispatch 2024-09-16T08:48:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[30137]: from='client.56876 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi160-75993-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:50.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[30137]: from='client.24730 ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"} : dispatch 2024-09-16T08:48:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[40028]: pgmap v168: 267 pgs: 267 active+clean; 600 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-09-16T08:48:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"} : dispatch 2024-09-16T08:48:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[40028]: from='client.24730 ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"} : dispatch 2024-09-16T08:48:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[40028]: from='client.56873 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi160-75757-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[40028]: from='client.24730 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pg_num","val":"11"}]': finished 2024-09-16T08:48:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[40028]: osdmap e148: 8 total, 8 up, 8 in 2024-09-16T08:48:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2515142612' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi160-75993-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:50.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"} : dispatch 2024-09-16T08:48:50.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[40028]: from='client.56876 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi160-75993-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:50.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:50 smithi160 ceph-mon[40028]: from='client.24730 ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"} : dispatch 2024-09-16T08:48:52.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:52 smithi160 ceph-mon[40028]: pgmap v171: 339 pgs: 72 unknown, 267 active+clean; 600 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-09-16T08:48:52.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:52 smithi160 ceph-mon[40028]: from='client.56876 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi160-75993-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:52.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:52 smithi160 ceph-mon[40028]: from='client.24730 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"}]': finished 2024-09-16T08:48:52.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:52.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:52 smithi160 ceph-mon[40028]: osdmap e149: 8 total, 8 up, 8 in 2024-09-16T08:48:52.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi160-75741-27"} : dispatch 2024-09-16T08:48:52.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:52 smithi160 ceph-mon[30137]: pgmap v171: 339 pgs: 72 unknown, 267 active+clean; 600 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-09-16T08:48:52.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:52 smithi160 ceph-mon[30137]: from='client.56876 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi160-75993-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:52.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:52 smithi160 ceph-mon[30137]: from='client.24730 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"}]': finished 2024-09-16T08:48:52.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:52.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:52 smithi160 ceph-mon[30137]: osdmap e149: 8 total, 8 up, 8 in 2024-09-16T08:48:52.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi160-75741-27"} : dispatch 2024-09-16T08:48:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:52 smithi169 ceph-mon[33465]: pgmap v171: 339 pgs: 72 unknown, 267 active+clean; 600 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-09-16T08:48:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:52 smithi169 ceph-mon[33465]: from='client.56876 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi160-75993-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:52 smithi169 ceph-mon[33465]: from='client.24730 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi160-75864-1","var":"pgp_num","val":"11"}]': finished 2024-09-16T08:48:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/454516762' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-16T08:48:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:52 smithi169 ceph-mon[33465]: osdmap e149: 8 total, 8 up, 8 in 2024-09-16T08:48:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi160-75741-27"} : dispatch 2024-09-16T08:48:53.183 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: Running main() from gmock_main.cc 2024-09-16T08:48:53.183 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-09-16T08:48:53.183 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [----------] Global test environment set-up. 2024-09-16T08:48:53.183 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-09-16T08:48:53.183 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-09-16T08:48:53.183 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.TooBig (2656 ms) 2024-09-16T08:48:53.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-09-16T08:48:53.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (3020 ms) 2024-09-16T08:48:53.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-09-16T08:48:53.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (3024 ms) 2024-09-16T08:48:53.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-09-16T08:48:53.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (2896 ms) 2024-09-16T08:48:53.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-09-16T08:48:53.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3017 ms) 2024-09-16T08:48:53.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-09-16T08:48:53.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (2999 ms) 2024-09-16T08:48:53.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-09-16T08:48:53.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3046 ms) 2024-09-16T08:48:53.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-09-16T08:48:53.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (3024 ms) 2024-09-16T08:48:53.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-09-16T08:48:53.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (2919 ms) 2024-09-16T08:48:53.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-09-16T08:48:53.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3055 ms) 2024-09-16T08:48:53.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-09-16T08:48:53.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (2983 ms) 2024-09-16T08:48:53.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-09-16T08:48:53.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3018 ms) 2024-09-16T08:48:53.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-09-16T08:48:53.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (2998 ms) 2024-09-16T08:48:53.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-09-16T08:48:53.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2950 ms) 2024-09-16T08:48:53.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-09-16T08:48:53.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.Flush (3014 ms) 2024-09-16T08:48:53.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-09-16T08:48:53.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (2997 ms) 2024-09-16T08:48:53.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-09-16T08:48:53.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (2989 ms) 2024-09-16T08:48:53.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-09-16T08:48:53.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3011 ms) 2024-09-16T08:48:53.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-09-16T08:48:53.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3011 ms) 2024-09-16T08:48:53.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-09-16T08:48:53.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3014 ms) 2024-09-16T08:48:53.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-09-16T08:48:53.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (3006 ms) 2024-09-16T08:48:53.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-09-16T08:48:53.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (2969 ms) 2024-09-16T08:48:53.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-09-16T08:48:53.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3007 ms) 2024-09-16T08:48:53.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-09-16T08:48:53.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3008 ms) 2024-09-16T08:48:53.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-09-16T08:48:53.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3010 ms) 2024-09-16T08:48:53.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-09-16T08:48:53.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3012 ms) 2024-09-16T08:48:53.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [----------] 26 tests from LibRadosAio (77653 ms total) 2024-09-16T08:48:53.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: 2024-09-16T08:48:53.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-09-16T08:48:53.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-09-16T08:48:53.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi160-75741-27"}]': finished 2024-09-16T08:48:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[30137]: osdmap e150: 8 total, 8 up, 8 in 2024-09-16T08:48:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi160-75741-27"} : dispatch 2024-09-16T08:48:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2158997020' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi160-75757-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[30137]: from='client.48052 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi160-75757-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[30137]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[30137]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi160-75864-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[30137]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi160-75864-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:48:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi160-75741-27"}]': finished 2024-09-16T08:48:53.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi160-75741-27"}]': finished 2024-09-16T08:48:53.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[40028]: osdmap e150: 8 total, 8 up, 8 in 2024-09-16T08:48:53.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi160-75741-27"} : dispatch 2024-09-16T08:48:53.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2158997020' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi160-75757-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:53.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[40028]: from='client.48052 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi160-75757-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:53.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:53.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[40028]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:53.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:53.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[40028]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:53.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi160-75864-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:53.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[40028]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi160-75864-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:53.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:48:53.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi160-75741-27"}]': finished 2024-09-16T08:48:53.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi160-75741-27"}]': finished 2024-09-16T08:48:53.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:53 smithi169 ceph-mon[33465]: osdmap e150: 8 total, 8 up, 8 in 2024-09-16T08:48:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi160-75741-27"} : dispatch 2024-09-16T08:48:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2158997020' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi160-75757-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:53 smithi169 ceph-mon[33465]: from='client.48052 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi160-75757-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:53 smithi169 ceph-mon[33465]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:53 smithi169 ceph-mon[33465]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi160-75864-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:53 smithi169 ceph-mon[33465]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi160-75864-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:48:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1145989171' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi160-75741-27"}]': finished 2024-09-16T08:48:54.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: pgmap v174: 268 pgs: 32 unknown, 236 active+clean; 598 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:48:54.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: from='client.48052 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi160-75757-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:54.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: from='client.48064 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi160-75864-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:54.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:48:54.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: osdmap e151: 8 total, 8 up, 8 in 2024-09-16T08:48:54.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi160-75864-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:54.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-13"} : dispatch 2024-09-16T08:48:54.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2452920335' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi160-75993-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:54.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi160-75864-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:54.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:54.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:54.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:54.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:54.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi160-75741-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:54.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi160-75741-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:54.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:54.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:48:54.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: pgmap v174: 268 pgs: 32 unknown, 236 active+clean; 598 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:48:54.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: from='client.48052 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi160-75757-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:54.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: from='client.48064 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi160-75864-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:54.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:48:54.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: osdmap e151: 8 total, 8 up, 8 in 2024-09-16T08:48:54.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi160-75864-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:54.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-13"} : dispatch 2024-09-16T08:48:54.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2452920335' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi160-75993-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:54.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi160-75864-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:54.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:54.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:54.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:54.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:54.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi160-75741-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:54.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi160-75741-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:54.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:54.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:48:54.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: pgmap v174: 268 pgs: 32 unknown, 236 active+clean; 598 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:48:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: from='client.48052 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi160-75757-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: from='client.48064 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi160-75864-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:48:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: osdmap e151: 8 total, 8 up, 8 in 2024-09-16T08:48:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi160-75864-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-13"} : dispatch 2024-09-16T08:48:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2452920335' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi160-75993-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi160-75864-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:54.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:54.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi160-75741-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:54.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi160-75741-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:54.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:54.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:48:55.209 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:48:54 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:48:55.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:48:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:48:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:48:55.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-13"}]': finished 2024-09-16T08:48:55.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2452920335' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi160-75993-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:55.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:55 smithi160 ceph-mon[30137]: from='client.48070 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi160-75741-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:55.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi160-75741-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:55.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:55 smithi160 ceph-mon[30137]: osdmap e152: 8 total, 8 up, 8 in 2024-09-16T08:48:55.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:55 smithi160 ceph-mon[30137]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi160-75741-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:55.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-13"}]': finished 2024-09-16T08:48:55.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2452920335' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi160-75993-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:55.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:55 smithi160 ceph-mon[40028]: from='client.48070 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi160-75741-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:55.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi160-75741-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:55.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:55 smithi160 ceph-mon[40028]: osdmap e152: 8 total, 8 up, 8 in 2024-09-16T08:48:55.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:55 smithi160 ceph-mon[40028]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi160-75741-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:55.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-13"}]': finished 2024-09-16T08:48:55.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2452920335' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi160-75993-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:55.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:55 smithi169 ceph-mon[33465]: from='client.48070 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi160-75741-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:55.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi160-75741-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:55.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:55 smithi169 ceph-mon[33465]: osdmap e152: 8 total, 8 up, 8 in 2024-09-16T08:48:55.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:55 smithi169 ceph-mon[33465]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi160-75741-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:56.251 INFO:tasks.workunit.client.0.smithi160.stdout: d=7 2024-09-16T08:48:56.251 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:f905c69b:::2:head 2024-09-16T08:48:56.251 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=14:f905c69b:::2:head expected=14:f905c69b:::2:head 2024-09-16T08:48:56.251 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:f905c69b:::2:head -> 2 2024-09-16T08:48:56.251 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=2 expected=2 2024-09-16T08:48:56.252 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:e960b815:::9:head 2024-09-16T08:48:56.252 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=14:e960b815:::9:head expected=14:e960b815:::9:head 2024-09-16T08:48:56.252 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:e960b815:::9:head -> 9 2024-09-16T08:48:56.252 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=9 expected=9 2024-09-16T08:48:56.252 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:de5d7c5f:::12:head 2024-09-16T08:48:56.252 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=14:de5d7c5f:::12:head expected=14:de5d7c5f:::12:head 2024-09-16T08:48:56.252 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:de5d7c5f:::12:head -> 12 2024-09-16T08:48:56.252 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=12 expected=12 2024-09-16T08:48:56.252 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:566253c9:::13:head 2024-09-16T08:48:56.252 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=14:566253c9:::13:head expected=14:566253c9:::13:head 2024-09-16T08:48:56.252 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:566253c9:::13:head -> 13 2024-09-16T08:48:56.252 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=13 expected=13 2024-09-16T08:48:56.253 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:89d3ae78:::11:head 2024-09-16T08:48:56.253 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=14:89d3ae78:::11:head expected=14:89d3ae78:::11:head 2024-09-16T08:48:56.253 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:89d3ae78:::11:head -> 11 2024-09-16T08:48:56.253 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=11 expected=11 2024-09-16T08:48:56.253 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:863748b0:::15:head 2024-09-16T08:48:56.253 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=14:863748b0:::15:head expected=14:863748b0:::15:head 2024-09-16T08:48:56.253 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:863748b0:::15:head -> 15 2024-09-16T08:48:56.253 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=15 expected=15 2024-09-16T08:48:56.253 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:6cac518f:::0:head 2024-09-16T08:48:56.253 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=14:6cac518f:::0:head expected=14:6cac518f:::0:head 2024-09-16T08:48:56.253 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:6cac518f:::0:head -> 0 2024-09-16T08:48:56.253 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=0 expected=0 2024-09-16T08:48:56.253 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 14:62a1935d:::14:head 2024-09-16T08:48:56.253 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=14:62a1935d:::14:head expected=14:62a1935d:::14:head 2024-09-16T08:48:56.254 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 14:62a1935d:::14:head -> 14 2024-09-16T08:48:56.254 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=14 expected=14 2024-09-16T08:48:56.254 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (211 ms) 2024-09-16T08:48:56.254 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-09-16T08:48:56.254 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (50198 ms) 2024-09-16T08:48:56.254 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-09-16T08:48:56.254 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: split 0/5 -> MIN 14:33333333::::head 2024-09-16T08:48:56.254 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: split 1/5 -> 14:33333333::::head 14:66666666::::head 2024-09-16T08:48:56.254 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: split 2/5 -> 14:66666666::::head 14:99999999::::head 2024-09-16T08:48:56.254 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: split 3/5 -> 14:99999999::::head 14:cccccccc::::head 2024-09-16T08:48:56.254 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: split 4/5 -> 14:cccccccc::::head MAX 2024-09-16T08:48:56.254 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (30053 ms) 2024-09-16T08:48:56.254 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [----------] 7 tests from LibRadosList (81114 ms total) 2024-09-16T08:48:56.254 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 2024-09-16T08:48:56.254 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-09-16T08:48:56.254 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-09-16T08:48:56.255 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1038 ms) 2024-09-16T08:48:56.255 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-09-16T08:48:56.255 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: myset foo1,foo2,foo3 2024-09-16T08:48:56.255 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo1 2024-09-16T08:48:56.255 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo2 2024-09-16T08:48:56.255 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo3 2024-09-16T08:48:56.255 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: myset foo1,foo4,foo5 2024-09-16T08:48:56.255 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo4 2024-09-16T08:48:56.255 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo5 2024-09-16T08:48:56.255 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo1 2024-09-16T08:48:56.255 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: myset foo6,foo7 2024-09-16T08:48:56.256 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo7 2024-09-16T08:48:56.256 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo6 2024-09-16T08:48:56.256 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-16T08:48:56.256 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns1:foo4 2024-09-16T08:48:56.256 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns1:foo5 2024-09-16T08:48:56.256 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns2:foo7 2024-09-16T08:48:56.256 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns2:foo6 2024-09-16T08:48:56.256 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns1:foo1 2024-09-16T08:48:56.256 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: :foo1 2024-09-16T08:48:56.256 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: :foo2 2024-09-16T08:48:56.256 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: :foo3 2024-09-16T08:48:56.256 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (30 ms) 2024-09-16T08:48:56.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:56 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:56 smithi169 ceph-mon[33465]: pgmap v177: 268 pgs: 32 creating+peering, 236 active+clean; 608 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 9.2 KiB/s rd, 2 op/s 2024-09-16T08:48:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:56 smithi169 ceph-mon[33465]: from='client.48064 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi160-75864-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi160-75864-2"}]': finished 2024-09-16T08:48:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:56 smithi169 ceph-mon[33465]: osdmap e153: 8 total, 8 up, 8 in 2024-09-16T08:48:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2953693594' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi160-75757-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:56 smithi169 ceph-mon[33465]: from='client.48076 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi160-75757-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:56 smithi169 ceph-mon[33465]: from='client.48070 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi160-75741-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi160-75741-28"}]': finished 2024-09-16T08:48:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:56 smithi169 ceph-mon[33465]: from='client.48076 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi160-75757-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:56 smithi169 ceph-mon[33465]: osdmap e154: 8 total, 8 up, 8 in 2024-09-16T08:48:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/581271744' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi160-75993-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:56.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:56 smithi169 ceph-mon[33465]: from='client.56909 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi160-75993-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[30137]: pgmap v177: 268 pgs: 32 creating+peering, 236 active+clean; 608 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 9.2 KiB/s rd, 2 op/s 2024-09-16T08:48:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[30137]: from='client.48064 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi160-75864-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi160-75864-2"}]': finished 2024-09-16T08:48:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[30137]: osdmap e153: 8 total, 8 up, 8 in 2024-09-16T08:48:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2953693594' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi160-75757-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[30137]: from='client.48076 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi160-75757-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[30137]: from='client.48070 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi160-75741-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi160-75741-28"}]': finished 2024-09-16T08:48:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[30137]: from='client.48076 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi160-75757-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[30137]: osdmap e154: 8 total, 8 up, 8 in 2024-09-16T08:48:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/581271744' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi160-75993-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:56.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[30137]: from='client.56909 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi160-75993-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:48:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[40028]: pgmap v177: 268 pgs: 32 creating+peering, 236 active+clean; 608 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 9.2 KiB/s rd, 2 op/s 2024-09-16T08:48:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[40028]: from='client.48064 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi160-75864-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi160-75864-2"}]': finished 2024-09-16T08:48:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[40028]: osdmap e153: 8 total, 8 up, 8 in 2024-09-16T08:48:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2953693594' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi160-75757-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[40028]: from='client.48076 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi160-75757-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[40028]: from='client.48070 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi160-75741-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi160-75741-28"}]': finished 2024-09-16T08:48:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[40028]: from='client.48076 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi160-75757-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[40028]: osdmap e154: 8 total, 8 up, 8 in 2024-09-16T08:48:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:56.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/581271744' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi160-75993-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:56.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:56 smithi160 ceph-mon[40028]: from='client.56909 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi160-75993-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:58.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:58 smithi160 ceph-mon[30137]: pgmap v180: 316 pgs: 112 unknown, 204 active+clean; 608 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-09-16T08:48:58.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:58.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:58 smithi160 ceph-mon[30137]: from='client.56909 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi160-75993-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:58.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:58 smithi160 ceph-mon[30137]: osdmap e155: 8 total, 8 up, 8 in 2024-09-16T08:48:58.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:58.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:58 smithi160 ceph-mon[30137]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:58.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:58 smithi160 ceph-mon[40028]: pgmap v180: 316 pgs: 112 unknown, 204 active+clean; 608 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-09-16T08:48:58.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:58.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:58 smithi160 ceph-mon[40028]: from='client.56909 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi160-75993-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:58.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:58 smithi160 ceph-mon[40028]: osdmap e155: 8 total, 8 up, 8 in 2024-09-16T08:48:58.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:58.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:58 smithi160 ceph-mon[40028]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:58.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:58 smithi169 ceph-mon[33465]: pgmap v180: 316 pgs: 112 unknown, 204 active+clean; 608 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-09-16T08:48:58.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:58.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:58 smithi169 ceph-mon[33465]: from='client.56909 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi160-75993-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:58.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:58 smithi169 ceph-mon[33465]: osdmap e155: 8 total, 8 up, 8 in 2024-09-16T08:48:58.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:58.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:58 smithi169 ceph-mon[33465]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.48064 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi160-75864-2"}]': finished 2024-09-16T08:48:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: osdmap e156: 8 total, 8 up, 8 in 2024-09-16T08:48:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1546929229' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi160-75757-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:59.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:59.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.56915 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi160-75757-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:59.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:59.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:48:59.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:59.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:59.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:59.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.48064 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi160-75864-2"}]': finished 2024-09-16T08:48:59.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.56915 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi160-75757-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:59.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.48070 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi160-75741-28"}]': finished 2024-09-16T08:48:59.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:48:59.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:59.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: osdmap e157: 8 total, 8 up, 8 in 2024-09-16T08:48:59.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:59.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-15"} : dispatch 2024-09-16T08:48:59.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:59.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi160-75993-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.603 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:48:59 smithi169 ceph-mon[33465]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi160-75993-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.48064 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi160-75864-2"}]': finished 2024-09-16T08:48:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: osdmap e156: 8 total, 8 up, 8 in 2024-09-16T08:48:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1546929229' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi160-75757-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:59.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.56915 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi160-75757-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:59.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:59.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:48:59.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:59.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:59.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:59.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.48064 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi160-75864-2"}]': finished 2024-09-16T08:48:59.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.56915 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi160-75757-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:59.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.48070 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi160-75741-28"}]': finished 2024-09-16T08:48:59.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.48064 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi160-75864-2"}]': finished 2024-09-16T08:48:59.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2352805635' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:59.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: osdmap e156: 8 total, 8 up, 8 in 2024-09-16T08:48:59.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1546929229' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi160-75757-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:59.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.48064 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi160-75864-2"} : dispatch 2024-09-16T08:48:59.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:59.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.56915 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi160-75757-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:48:59.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:59.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:48:59.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:59.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:48:59.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:48:59.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.48064 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi160-75864-2"}]': finished 2024-09-16T08:48:59.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.56915 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi160-75757-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:48:59.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.48070 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi160-75741-28"}]': finished 2024-09-16T08:48:59.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:48:59.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:59.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: osdmap e157: 8 total, 8 up, 8 in 2024-09-16T08:48:59.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:59.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-15"} : dispatch 2024-09-16T08:48:59.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:59.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi160-75993-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[30137]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi160-75993-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:48:59.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:48:59.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: osdmap e157: 8 total, 8 up, 8 in 2024-09-16T08:48:59.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/704732976' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:59.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-15"} : dispatch 2024-09-16T08:48:59.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.48070 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi160-75741-28"} : dispatch 2024-09-16T08:48:59.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi160-75993-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:48:59.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:48:59 smithi160 ceph-mon[40028]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi160-75993-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:49:00.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:00 smithi169 ceph-mon[33465]: pgmap v183: 268 pgs: 32 creating+peering, 236 active+clean; 608 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-15"}]': finished 2024-09-16T08:49:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:00 smithi169 ceph-mon[33465]: from='client.48070 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi160-75741-28"}]': finished 2024-09-16T08:49:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:49:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:00 smithi169 ceph-mon[33465]: osdmap e158: 8 total, 8 up, 8 in 2024-09-16T08:49:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-15", "mode": "writeback"} : dispatch 2024-09-16T08:49:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:00 smithi169 ceph-mon[33465]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:49:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1655427236' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75864-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:00 smithi169 ceph-mon[33465]: from='client.48100 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75864-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi160-75741-29"} : dispatch 2024-09-16T08:49:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi160-75741-29"} : dispatch 2024-09-16T08:49:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi160-75741-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[30137]: pgmap v183: 268 pgs: 32 creating+peering, 236 active+clean; 608 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-15"}]': finished 2024-09-16T08:49:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[30137]: from='client.48070 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi160-75741-28"}]': finished 2024-09-16T08:49:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:49:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[30137]: osdmap e158: 8 total, 8 up, 8 in 2024-09-16T08:49:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-15", "mode": "writeback"} : dispatch 2024-09-16T08:49:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[30137]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:49:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1655427236' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75864-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[30137]: from='client.48100 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75864-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi160-75741-29"} : dispatch 2024-09-16T08:49:00.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi160-75741-29"} : dispatch 2024-09-16T08:49:00.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi160-75741-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[40028]: pgmap v183: 268 pgs: 32 creating+peering, 236 active+clean; 608 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-15"}]': finished 2024-09-16T08:49:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[40028]: from='client.48070 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi160-75741-28"}]': finished 2024-09-16T08:49:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:49:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[40028]: osdmap e158: 8 total, 8 up, 8 in 2024-09-16T08:49:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-15", "mode": "writeback"} : dispatch 2024-09-16T08:49:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[40028]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:49:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1655427236' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75864-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[40028]: from='client.48100 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75864-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:00.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi160-75741-29"} : dispatch 2024-09-16T08:49:00.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi160-75741-29"} : dispatch 2024-09-16T08:49:00.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi160-75741-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:01.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:01 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:01.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:01 smithi169 ceph-mon[33465]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi160-75993-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi160-75993-36"}]': finished 2024-09-16T08:49:01.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-15", "mode": "writeback"}]': finished 2024-09-16T08:49:01.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:01 smithi169 ceph-mon[33465]: from='client.48022 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"}]': finished 2024-09-16T08:49:01.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:01 smithi169 ceph-mon[33465]: from='client.48100 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75864-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:01.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi160-75741-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:01.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:49:01.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:01 smithi169 ceph-mon[33465]: osdmap e159: 8 total, 8 up, 8 in 2024-09-16T08:49:01.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi160-75741-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi160-75741-29"} : dispatch 2024-09-16T08:49:01.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:01 smithi169 ceph-mon[33465]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:49:01.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4031774228' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi160-75757-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:01.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1655427236' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi160-75864-3","pool2":"test-rados-api-smithi160-75864-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-16T08:49:01.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:01 smithi169 ceph-mon[33465]: from='client.48100 ' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi160-75864-3","pool2":"test-rados-api-smithi160-75864-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-16T08:49:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[30137]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi160-75993-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi160-75993-36"}]': finished 2024-09-16T08:49:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-15", "mode": "writeback"}]': finished 2024-09-16T08:49:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[30137]: from='client.48022 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"}]': finished 2024-09-16T08:49:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[30137]: from='client.48100 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75864-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi160-75741-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:49:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[30137]: osdmap e159: 8 total, 8 up, 8 in 2024-09-16T08:49:01.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi160-75741-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi160-75741-29"} : dispatch 2024-09-16T08:49:01.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[30137]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:49:01.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4031774228' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi160-75757-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:01.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1655427236' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi160-75864-3","pool2":"test-rados-api-smithi160-75864-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-16T08:49:01.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[30137]: from='client.48100 ' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi160-75864-3","pool2":"test-rados-api-smithi160-75864-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-16T08:49:01.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:01.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[40028]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi160-75993-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi160-75993-36"}]': finished 2024-09-16T08:49:01.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-15", "mode": "writeback"}]': finished 2024-09-16T08:49:01.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[40028]: from='client.48022 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"}]': finished 2024-09-16T08:49:01.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[40028]: from='client.48100 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-75864-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:01.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi160-75741-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:01.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4061553337' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:49:01.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[40028]: osdmap e159: 8 total, 8 up, 8 in 2024-09-16T08:49:01.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi160-75741-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi160-75741-29"} : dispatch 2024-09-16T08:49:01.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[40028]: from='client.48022 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"} : dispatch 2024-09-16T08:49:01.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4031774228' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi160-75757-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:01.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1655427236' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi160-75864-3","pool2":"test-rados-api-smithi160-75864-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-16T08:49:01.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:01 smithi160 ceph-mon[40028]: from='client.48100 ' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi160-75864-3","pool2":"test-rados-api-smithi160-75864-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-16T08:49:02.145 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosListEC.LisSnapshotsECPP.SnapRemovePP (2005 ms) 2024-09-16T08:49:02.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-09-16T08:49:02.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2006 ms) 2024-09-16T08:49:02.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-09-16T08:49:02.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2012 ms) 2024-09-16T08:49:02.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8994 ms total) 2024-09-16T08:49:02.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: 2024-09-16T08:49:02.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-09-16T08:49:02.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-09-16T08:49:02.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4051 ms) 2024-09-16T08:49:02.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-09-16T08:49:02.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3979 ms) 2024-09-16T08:49:02.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-09-16T08:49:02.147 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3944 ms) 2024-09-16T08:49:02.147 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11974 ms total) 2024-09-16T08:49:02.147 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: 2024-09-16T08:49:02.147 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-09-16T08:49:02.147 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (93496 ms total) 2024-09-16T08:49:02.147 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-09-16T08:49:02.147 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-09-16T08:49:02.147 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-09-16T08:49:02.147 INFO:tasks.workunit.client.0.smithi160.stdout:tObjectsStart 2024-09-16T08:49:02.147 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 1 0 2024-09-16T08:49:02.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 10 0 2024-09-16T08:49:02.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 13 0 2024-09-16T08:49:02.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 7 0 2024-09-16T08:49:02.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 14 0 2024-09-16T08:49:02.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 0 0 2024-09-16T08:49:02.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 15 0 2024-09-16T08:49:02.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 11 0 2024-09-16T08:49:02.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 5 0 2024-09-16T08:49:02.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 8 0 2024-09-16T08:49:02.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 6 0 2024-09-16T08:49:02.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 3 0 2024-09-16T08:49:02.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 4 0 2024-09-16T08:49:02.148 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 12 0 2024-09-16T08:49:02.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 9 0 2024-09-16T08:49:02.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 2 0 2024-09-16T08:49:02.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-16T08:49:02.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (42 ms) 2024-09-16T08:49:02.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [----------] 3 tests from LibRadosListEC (1110 ms total) 2024-09-16T08:49:02.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 2024-09-16T08:49:02.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-09-16T08:49:02.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-09-16T08:49:02.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3013 ms) 2024-09-16T08:49:02.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [----------] 1 test from LibRadosListNP (3013 ms total) 2024-09-16T08:49:02.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 2024-09-16T08:49:02.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [----------] Global test environment tear-down 2024-09-16T08:49:02.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [==========] 11 tests from 3 test suites ran. (93554 ms total) 2024-09-16T08:49:02.149 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ PASSED ] 11 tests. 2024-09-16T08:49:02.501 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:02 smithi169 ceph-mon[33465]: pgmap v186: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:02.501 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:02 smithi169 ceph-mon[33465]: from='client.48022 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"}]': finished 2024-09-16T08:49:02.501 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:02 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4031774228' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi160-75757-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:02.501 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:02 smithi169 ceph-mon[33465]: from='client.48100 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi160-75864-3","pool2":"test-rados-api-smithi160-75864-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-16T08:49:02.501 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:02 smithi169 ceph-mon[33465]: osdmap e160: 8 total, 8 up, 8 in 2024-09-16T08:49:02.501 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:02 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:02 smithi160 ceph-mon[30137]: pgmap v186: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:02 smithi160 ceph-mon[30137]: from='client.48022 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"}]': finished 2024-09-16T08:49:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:02 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4031774228' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi160-75757-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:02 smithi160 ceph-mon[30137]: from='client.48100 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi160-75864-3","pool2":"test-rados-api-smithi160-75864-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-16T08:49:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:02 smithi160 ceph-mon[30137]: osdmap e160: 8 total, 8 up, 8 in 2024-09-16T08:49:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:02 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:02.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:02 smithi160 ceph-mon[40028]: pgmap v186: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:02.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:02 smithi160 ceph-mon[40028]: from='client.48022 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi160-76133-21"}]': finished 2024-09-16T08:49:02.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:02 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4031774228' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi160-75757-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:02.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:02 smithi160 ceph-mon[40028]: from='client.48100 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi160-75864-3","pool2":"test-rados-api-smithi160-75864-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-16T08:49:02.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:02 smithi160 ceph-mon[40028]: osdmap e160: 8 total, 8 up, 8 in 2024-09-16T08:49:02.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:02 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:04.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[30137]: pgmap v189: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:04.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi160-75741-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi160-75741-29"}]': finished 2024-09-16T08:49:04.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:04.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[30137]: osdmap e161: 8 total, 8 up, 8 in 2024-09-16T08:49:04.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-15"} : dispatch 2024-09-16T08:49:04.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:49:04.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[30137]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:49:04.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:49:04.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:04.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[40028]: pgmap v189: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:04.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi160-75741-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi160-75741-29"}]': finished 2024-09-16T08:49:04.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:04.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[40028]: osdmap e161: 8 total, 8 up, 8 in 2024-09-16T08:49:04.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-15"} : dispatch 2024-09-16T08:49:04.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:49:04.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[40028]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:49:04.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:49:04.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:04.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:04 smithi169 ceph-mon[33465]: pgmap v189: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi160-75741-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi160-75741-29"}]': finished 2024-09-16T08:49:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:04 smithi169 ceph-mon[33465]: osdmap e161: 8 total, 8 up, 8 in 2024-09-16T08:49:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-15"} : dispatch 2024-09-16T08:49:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:49:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:04 smithi169 ceph-mon[33465]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:49:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:49:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:04 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:05.154 INFO:tasks.workunit.client.0.smithi160.stdout: (67992 ms total) 2024-09-16T08:49:05.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: 2024-09-16T08:49:05.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-09-16T08:49:05.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-09-16T08:49:05.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (109 ms) 2024-09-16T08:49:05.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (109 ms total) 2024-09-16T08:49:05.154 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: 2024-09-16T08:49:05.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-09-16T08:49:05.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-09-16T08:49:05.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (33 ms) 2024-09-16T08:49:05.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-09-16T08:49:05.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (8 ms) 2024-09-16T08:49:05.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (41 ms total) 2024-09-16T08:49:05.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: 2024-09-16T08:49:05.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-09-16T08:49:05.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-09-16T08:49:05.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (9 ms) 2024-09-16T08:49:05.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-09-16T08:49:05.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (2 ms) 2024-09-16T08:49:05.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (11 ms total) 2024-09-16T08:49:05.155 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: 2024-09-16T08:49:05.156 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-09-16T08:49:05.156 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-09-16T08:49:05.156 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (47 ms) 2024-09-16T08:49:05.156 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-09-16T08:49:05.156 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (3 ms) 2024-09-16T08:49:05.156 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (50 ms total) 2024-09-16T08:49:05.156 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: 2024-09-16T08:49:05.156 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-09-16T08:49:05.156 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-09-16T08:49:05.156 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1044 ms) 2024-09-16T08:49:05.156 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1044 ms total) 2024-09-16T08:49:05.156 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: 2024-09-16T08:49:05.156 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-09-16T08:49:05.157 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (96534 ms total) 2024-09-16T08:49:05.157 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-09-16T08:49:05.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:49:04 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:49:05.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:49:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:49:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:49:05.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:05.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-15"}]': finished 2024-09-16T08:49:05.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[40028]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36"}]': finished 2024-09-16T08:49:05.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[40028]: osdmap e162: 8 total, 8 up, 8 in 2024-09-16T08:49:05.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:49:05.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3789378908' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi160-75757-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:05.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[40028]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:49:05.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[40028]: from='client.56945 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi160-75757-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:05.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[40028]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi160-75993-36"}]': finished 2024-09-16T08:49:05.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[40028]: from='client.56945 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi160-75757-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:05.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[40028]: osdmap e163: 8 total, 8 up, 8 in 2024-09-16T08:49:05.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi160-75741-29"} : dispatch 2024-09-16T08:49:05.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:05.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-15"}]': finished 2024-09-16T08:49:05.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[30137]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36"}]': finished 2024-09-16T08:49:05.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[30137]: osdmap e162: 8 total, 8 up, 8 in 2024-09-16T08:49:05.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:49:05.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3789378908' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi160-75757-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:05.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[30137]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:49:05.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[30137]: from='client.56945 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi160-75757-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:05.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[30137]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi160-75993-36"}]': finished 2024-09-16T08:49:05.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[30137]: from='client.56945 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi160-75757-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:05.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[30137]: osdmap e163: 8 total, 8 up, 8 in 2024-09-16T08:49:05.450 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi160-75741-29"} : dispatch 2024-09-16T08:49:05.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:05 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:05.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-15"}]': finished 2024-09-16T08:49:05.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:05 smithi169 ceph-mon[33465]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi160-75993-36"}]': finished 2024-09-16T08:49:05.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:05 smithi169 ceph-mon[33465]: osdmap e162: 8 total, 8 up, 8 in 2024-09-16T08:49:05.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3991204879' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:49:05.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3789378908' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi160-75757-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:05.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:05 smithi169 ceph-mon[33465]: from='client.56921 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi160-75993-36"} : dispatch 2024-09-16T08:49:05.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:05 smithi169 ceph-mon[33465]: from='client.56945 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi160-75757-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:05.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:05 smithi169 ceph-mon[33465]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi160-75993-36"}]': finished 2024-09-16T08:49:05.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:05 smithi169 ceph-mon[33465]: from='client.56945 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi160-75757-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:05.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:05 smithi169 ceph-mon[33465]: osdmap e163: 8 total, 8 up, 8 in 2024-09-16T08:49:05.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi160-75741-29"} : dispatch 2024-09-16T08:49:06.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:06.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:06 smithi160 ceph-mon[30137]: pgmap v192: 268 pgs: 32 unknown, 8 creating+peering, 228 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:49:06.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:06 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi160-75741-29"}]': finished 2024-09-16T08:49:06.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:06 smithi160 ceph-mon[30137]: osdmap e164: 8 total, 8 up, 8 in 2024-09-16T08:49:06.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:06 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi160-75741-29"} : dispatch 2024-09-16T08:49:06.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:06 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:06.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:06.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:06 smithi160 ceph-mon[40028]: pgmap v192: 268 pgs: 32 unknown, 8 creating+peering, 228 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:49:06.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:06 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi160-75741-29"}]': finished 2024-09-16T08:49:06.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:06 smithi160 ceph-mon[40028]: osdmap e164: 8 total, 8 up, 8 in 2024-09-16T08:49:06.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:06 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi160-75741-29"} : dispatch 2024-09-16T08:49:06.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:06 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:06.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:06 smithi169 ceph-mon[33465]: pgmap v192: 268 pgs: 32 unknown, 8 creating+peering, 228 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:49:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:06 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi160-75741-29"}]': finished 2024-09-16T08:49:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:06 smithi169 ceph-mon[33465]: osdmap e164: 8 total, 8 up, 8 in 2024-09-16T08:49:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:06 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi160-75741-29"} : dispatch 2024-09-16T08:49:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:06 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:08.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[30137]: pgmap v195: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:49:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi160-75741-29"}]': finished 2024-09-16T08:49:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[30137]: osdmap e165: 8 total, 8 up, 8 in 2024-09-16T08:49:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4180268395' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi160-75757-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[30137]: from='client.56951 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi160-75757-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[30137]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:08.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:08.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[30137]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:08.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi160-75741-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:08.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[30137]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi160-75741-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:08.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[40028]: pgmap v195: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:49:08.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi160-75741-29"}]': finished 2024-09-16T08:49:08.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:08.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[40028]: osdmap e165: 8 total, 8 up, 8 in 2024-09-16T08:49:08.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4180268395' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi160-75757-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:08.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[40028]: from='client.56951 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi160-75757-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:08.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:08.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:08.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[40028]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:08.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:08.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[40028]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:08.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi160-75741-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:08.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:08 smithi160 ceph-mon[40028]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi160-75741-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:08.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:08 smithi169 ceph-mon[33465]: pgmap v195: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:49:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/236707177' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi160-75741-29"}]': finished 2024-09-16T08:49:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:08 smithi169 ceph-mon[33465]: osdmap e165: 8 total, 8 up, 8 in 2024-09-16T08:49:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4180268395' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi160-75757-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:08 smithi169 ceph-mon[33465]: from='client.56951 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi160-75757-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:08 smithi169 ceph-mon[33465]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:08 smithi169 ceph-mon[33465]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi160-75741-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:08 smithi169 ceph-mon[33465]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi160-75741-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:09.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-09-16T08:49:09.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-09-16T08:49:09.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-09-16T08:49:09.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-09-16T08:49:09.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-09-16T08:49:09.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (2649 ms) 2024-09-16T08:49:09.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-09-16T08:49:09.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (20035 ms) 2024-09-16T08:49:09.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-09-16T08:49:09.142 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (5952 ms) 2024-09-16T08:49:09.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-09-16T08:49:09.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (3000 ms) 2024-09-16T08:49:09.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-09-16T08:49:09.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (2996 ms) 2024-09-16T08:49:09.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-09-16T08:49:09.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (3011 ms) 2024-09-16T08:49:09.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-09-16T08:49:09.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (2960 ms) 2024-09-16T08:49:09.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-09-16T08:49:09.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3011 ms) 2024-09-16T08:49:09.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-09-16T08:49:09.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2996 ms) 2024-09-16T08:49:09.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-09-16T08:49:09.144 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3002 ms) 2024-09-16T08:49:09.144 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-09-16T08:49:09.144 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (2996 ms) 2024-09-16T08:49:09.144 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-09-16T08:49:09.144 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3016 ms) 2024-09-16T08:49:09.144 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-09-16T08:49:09.144 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (2998 ms) 2024-09-16T08:49:09.144 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-09-16T08:49:09.144 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3009 ms) 2024-09-16T08:49:09.144 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-09-16T08:49:09.144 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3003 ms) 2024-09-16T08:49:09.144 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-09-16T08:49:09.144 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (2981 ms) 2024-09-16T08:49:09.145 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-09-16T08:49:09.145 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3008 ms) 2024-09-16T08:49:09.145 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-09-16T08:49:09.145 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3004 ms) 2024-09-16T08:49:09.145 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-09-16T08:49:09.145 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3009 ms) 2024-09-16T08:49:09.145 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-09-16T08:49:09.145 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3016 ms) 2024-09-16T08:49:09.145 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-09-16T08:49:09.145 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3000 ms) 2024-09-16T08:49:09.145 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-09-16T08:49:09.145 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3012 ms) 2024-09-16T08:49:09.145 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-09-16T08:49:09.145 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (2999 ms) 2024-09-16T08:49:09.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-09-16T08:49:09.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (2976 ms) 2024-09-16T08:49:09.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-09-16T08:49:09.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2985 ms) 2024-09-16T08:49:09.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-09-16T08:49:09.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3011 ms) 2024-09-16T08:49:09.146 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-09-16T08:49:09.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[40028]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi160-75757-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[40028]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi160-75741-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[40028]: osdmap e166: 8 total, 8 up, 8 in 2024-09-16T08:49:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi160-75741-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-17"} : dispatch 2024-09-16T08:49:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[40028]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi160-75741-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-17"}]': finished 2024-09-16T08:49:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[40028]: osdmap e167: 8 total, 8 up, 8 in 2024-09-16T08:49:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-17", "mode": "writeback"} : dispatch 2024-09-16T08:49:09.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[30137]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi160-75757-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:09.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:09.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[30137]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi160-75741-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:09.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[30137]: osdmap e166: 8 total, 8 up, 8 in 2024-09-16T08:49:09.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi160-75741-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:09.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-17"} : dispatch 2024-09-16T08:49:09.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[30137]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi160-75741-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:09.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-17"}]': finished 2024-09-16T08:49:09.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[30137]: osdmap e167: 8 total, 8 up, 8 in 2024-09-16T08:49:09.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-17", "mode": "writeback"} : dispatch 2024-09-16T08:49:09.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:09 smithi169 ceph-mon[33465]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi160-75757-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:09.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:09.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:09 smithi169 ceph-mon[33465]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi160-75741-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:09.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:09 smithi169 ceph-mon[33465]: osdmap e166: 8 total, 8 up, 8 in 2024-09-16T08:49:09.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi160-75741-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:09.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-17"} : dispatch 2024-09-16T08:49:09.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:09 smithi169 ceph-mon[33465]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi160-75741-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:09.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-17"}]': finished 2024-09-16T08:49:09.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:09 smithi169 ceph-mon[33465]: osdmap e167: 8 total, 8 up, 8 in 2024-09-16T08:49:09.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-17", "mode": "writeback"} : dispatch 2024-09-16T08:49:10.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:10 smithi160 ceph-mon[30137]: pgmap v198: 260 pgs: 260 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-16T08:49:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:10 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:10 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:10 smithi160 ceph-mon[30137]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi160-75741-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi160-75741-30"}]': finished 2024-09-16T08:49:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-17", "mode": "writeback"}]': finished 2024-09-16T08:49:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:10 smithi160 ceph-mon[30137]: osdmap e168: 8 total, 8 up, 8 in 2024-09-16T08:49:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/791697632' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi160-75757-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:10 smithi160 ceph-mon[30137]: from='client.48136 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi160-75757-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:10.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:10 smithi160 ceph-mon[40028]: pgmap v198: 260 pgs: 260 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-16T08:49:10.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:10 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:10.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:10 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:10.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:10 smithi160 ceph-mon[40028]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi160-75741-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi160-75741-30"}]': finished 2024-09-16T08:49:10.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-17", "mode": "writeback"}]': finished 2024-09-16T08:49:10.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:10 smithi160 ceph-mon[40028]: osdmap e168: 8 total, 8 up, 8 in 2024-09-16T08:49:10.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/791697632' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi160-75757-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:10.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:10 smithi160 ceph-mon[40028]: from='client.48136 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi160-75757-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:10.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:10 smithi169 ceph-mon[33465]: pgmap v198: 260 pgs: 260 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-16T08:49:10.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:10 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:10 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:10 smithi169 ceph-mon[33465]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi160-75741-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi160-75741-30"}]': finished 2024-09-16T08:49:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-17", "mode": "writeback"}]': finished 2024-09-16T08:49:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:10 smithi169 ceph-mon[33465]: osdmap e168: 8 total, 8 up, 8 in 2024-09-16T08:49:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/791697632' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi160-75757-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:10 smithi169 ceph-mon[33465]: from='client.48136 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi160-75757-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:11.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:11.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:11 smithi160 ceph-mon[30137]: from='client.48136 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi160-75757-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:11.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:11.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:11 smithi160 ceph-mon[30137]: osdmap e169: 8 total, 8 up, 8 in 2024-09-16T08:49:11.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-17"} : dispatch 2024-09-16T08:49:11.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:11.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:11 smithi160 ceph-mon[40028]: from='client.48136 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi160-75757-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:11.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:11.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:11 smithi160 ceph-mon[40028]: osdmap e169: 8 total, 8 up, 8 in 2024-09-16T08:49:11.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-17"} : dispatch 2024-09-16T08:49:11.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:11 smithi169 ceph-mon[33465]: from='client.48136 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi160-75757-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:11 smithi169 ceph-mon[33465]: osdmap e169: 8 total, 8 up, 8 in 2024-09-16T08:49:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-17"} : dispatch 2024-09-16T08:49:12.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:12 smithi160 ceph-mon[30137]: pgmap v201: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:49:12.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:12 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:12.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-17"}]': finished 2024-09-16T08:49:12.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:12 smithi160 ceph-mon[30137]: osdmap e170: 8 total, 8 up, 8 in 2024-09-16T08:49:12.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:12.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:12 smithi160 ceph-mon[30137]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:12 smithi160 ceph-mon[40028]: pgmap v201: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:49:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:12 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-17"}]': finished 2024-09-16T08:49:12.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:12 smithi160 ceph-mon[40028]: osdmap e170: 8 total, 8 up, 8 in 2024-09-16T08:49:12.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:12.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:12 smithi160 ceph-mon[40028]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:12.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:12 smithi169 ceph-mon[33465]: pgmap v201: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:49:12.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:12 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-17"}]': finished 2024-09-16T08:49:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:12 smithi169 ceph-mon[33465]: osdmap e170: 8 total, 8 up, 8 in 2024-09-16T08:49:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:12 smithi169 ceph-mon[33465]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:14.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:14 smithi160 ceph-mon[40028]: pgmap v204: 228 pgs: 228 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:14.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:14 smithi160 ceph-mon[40028]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi160-75741-30"}]': finished 2024-09-16T08:49:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:14 smithi160 ceph-mon[40028]: osdmap e171: 8 total, 8 up, 8 in 2024-09-16T08:49:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:14 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:14 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1895535571' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi160-75757-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:14 smithi160 ceph-mon[40028]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:14 smithi160 ceph-mon[40028]: from='client.56966 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi160-75757-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:14.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:14 smithi160 ceph-mon[30137]: pgmap v204: 228 pgs: 228 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:14.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:14 smithi160 ceph-mon[30137]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi160-75741-30"}]': finished 2024-09-16T08:49:14.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:14 smithi160 ceph-mon[30137]: osdmap e171: 8 total, 8 up, 8 in 2024-09-16T08:49:14.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:14.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1895535571' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi160-75757-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:14.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:14 smithi160 ceph-mon[30137]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:14.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:14 smithi160 ceph-mon[30137]: from='client.56966 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi160-75757-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:14.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:14 smithi169 ceph-mon[33465]: pgmap v204: 228 pgs: 228 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:14 smithi169 ceph-mon[33465]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi160-75741-30"}]': finished 2024-09-16T08:49:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:14 smithi169 ceph-mon[33465]: osdmap e171: 8 total, 8 up, 8 in 2024-09-16T08:49:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:14 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/788930389' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:14 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1895535571' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi160-75757-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:14 smithi169 ceph-mon[33465]: from='client.56957 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi160-75741-30"} : dispatch 2024-09-16T08:49:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:14 smithi169 ceph-mon[33465]: from='client.56966 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi160-75757-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:15.154 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:49:14 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:49:15.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:49:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:49:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:49:15.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[40028]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi160-75741-30"}]': finished 2024-09-16T08:49:15.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[40028]: from='client.56966 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi160-75757-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:15.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[40028]: osdmap e172: 8 total, 8 up, 8 in 2024-09-16T08:49:15.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:15.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:15.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[40028]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:15.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:15.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[40028]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:15.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi160-75741-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:15.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[40028]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi160-75741-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:15.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[30137]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi160-75741-30"}]': finished 2024-09-16T08:49:15.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[30137]: from='client.56966 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi160-75757-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:15.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[30137]: osdmap e172: 8 total, 8 up, 8 in 2024-09-16T08:49:15.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:15.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:15.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[30137]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:15.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:15.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[30137]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:15.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi160-75741-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:15.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:15 smithi160 ceph-mon[30137]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi160-75741-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:15.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:15 smithi169 ceph-mon[33465]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi160-75741-30"}]': finished 2024-09-16T08:49:15.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:15 smithi169 ceph-mon[33465]: from='client.56966 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi160-75757-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:15.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:15 smithi169 ceph-mon[33465]: osdmap e172: 8 total, 8 up, 8 in 2024-09-16T08:49:15.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:15 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:15.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:15 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:15.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:15 smithi169 ceph-mon[33465]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:15.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:15 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:15.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:15 smithi169 ceph-mon[33465]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:15.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:15 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi160-75741-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:15.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:15 smithi169 ceph-mon[33465]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi160-75741-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:16.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[30137]: pgmap v207: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:49:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[30137]: from='client.56972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi160-75741-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[30137]: osdmap e173: 8 total, 8 up, 8 in 2024-09-16T08:49:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi160-75741-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[30137]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi160-75741-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:16.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:16.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[40028]: pgmap v207: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:49:16.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:16.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:16.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[40028]: from='client.56972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi160-75741-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:16.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[40028]: osdmap e173: 8 total, 8 up, 8 in 2024-09-16T08:49:16.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi160-75741-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:16.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:16.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:16 smithi160 ceph-mon[40028]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi160-75741-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:16.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:16.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:16 smithi169 ceph-mon[33465]: pgmap v207: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:49:16.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:16 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:16 smithi169 ceph-mon[33465]: from='client.56972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi160-75741-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:16 smithi169 ceph-mon[33465]: osdmap e173: 8 total, 8 up, 8 in 2024-09-16T08:49:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi160-75741-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:16 smithi169 ceph-mon[33465]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi160-75741-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[30137]: osdmap e174: 8 total, 8 up, 8 in 2024-09-16T08:49:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-19"} : dispatch 2024-09-16T08:49:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3756803239' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi160-75757-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[30137]: from='client.56978 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi160-75757-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:17.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[30137]: from='client.56972 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi160-75741-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi160-75741-31"}]': finished 2024-09-16T08:49:17.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-19"}]': finished 2024-09-16T08:49:17.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[30137]: from='client.56978 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi160-75757-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:17.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[30137]: osdmap e175: 8 total, 8 up, 8 in 2024-09-16T08:49:17.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-19", "mode": "writeback"} : dispatch 2024-09-16T08:49:17.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:17.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[40028]: osdmap e174: 8 total, 8 up, 8 in 2024-09-16T08:49:17.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-19"} : dispatch 2024-09-16T08:49:17.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3756803239' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi160-75757-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:17.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[40028]: from='client.56978 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi160-75757-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:17.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[40028]: from='client.56972 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi160-75741-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi160-75741-31"}]': finished 2024-09-16T08:49:17.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-19"}]': finished 2024-09-16T08:49:17.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[40028]: from='client.56978 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi160-75757-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:17.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[40028]: osdmap e175: 8 total, 8 up, 8 in 2024-09-16T08:49:17.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-19", "mode": "writeback"} : dispatch 2024-09-16T08:49:17.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:17.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:17 smithi169 ceph-mon[33465]: osdmap e174: 8 total, 8 up, 8 in 2024-09-16T08:49:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-19"} : dispatch 2024-09-16T08:49:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3756803239' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi160-75757-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:17 smithi169 ceph-mon[33465]: from='client.56978 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi160-75757-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:17 smithi169 ceph-mon[33465]: from='client.56972 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi160-75741-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi160-75741-31"}]': finished 2024-09-16T08:49:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-19"}]': finished 2024-09-16T08:49:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:17 smithi169 ceph-mon[33465]: from='client.56978 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi160-75757-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:17 smithi169 ceph-mon[33465]: osdmap e175: 8 total, 8 up, 8 in 2024-09-16T08:49:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-19", "mode": "writeback"} : dispatch 2024-09-16T08:49:18.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:18 smithi160 ceph-mon[30137]: pgmap v210: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:49:18.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:18 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:18.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-19", "mode": "writeback"}]': finished 2024-09-16T08:49:18.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:18 smithi160 ceph-mon[30137]: osdmap e176: 8 total, 8 up, 8 in 2024-09-16T08:49:18.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:18 smithi160 ceph-mon[40028]: pgmap v210: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:49:18.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:18 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:18.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-19", "mode": "writeback"}]': finished 2024-09-16T08:49:18.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:18 smithi160 ceph-mon[40028]: osdmap e176: 8 total, 8 up, 8 in 2024-09-16T08:49:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:18 smithi169 ceph-mon[33465]: pgmap v210: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:49:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:18 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-19", "mode": "writeback"}]': finished 2024-09-16T08:49:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:18 smithi169 ceph-mon[33465]: osdmap e176: 8 total, 8 up, 8 in 2024-09-16T08:49:19.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:19.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:49:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:49:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[30137]: osdmap e177: 8 total, 8 up, 8 in 2024-09-16T08:49:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/58716442' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi160-75757-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-19"} : dispatch 2024-09-16T08:49:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[30137]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[30137]: from='client.56984 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi160-75757-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:19.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:19.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:49:19.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:49:19.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:19.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[40028]: osdmap e177: 8 total, 8 up, 8 in 2024-09-16T08:49:19.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:19.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/58716442' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi160-75757-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:19.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-19"} : dispatch 2024-09-16T08:49:19.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[40028]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:19.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:19 smithi160 ceph-mon[40028]: from='client.56984 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi160-75757-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:19.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:19.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:49:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:49:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:19 smithi169 ceph-mon[33465]: osdmap e177: 8 total, 8 up, 8 in 2024-09-16T08:49:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/58716442' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi160-75757-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-19"} : dispatch 2024-09-16T08:49:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:19 smithi169 ceph-mon[33465]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:19 smithi169 ceph-mon[33465]: from='client.56984 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi160-75757-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:20.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[30137]: pgmap v213: 236 pgs: 236 active+clean; 596 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 4.5 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2024-09-16T08:49:20.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:49:20.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:49:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:49:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-19"}]': finished 2024-09-16T08:49:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[30137]: from='client.56972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi160-75741-31"}]': finished 2024-09-16T08:49:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[30137]: from='client.56984 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi160-75757-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[30137]: osdmap e178: 8 total, 8 up, 8 in 2024-09-16T08:49:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[30137]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:20.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[40028]: pgmap v213: 236 pgs: 236 active+clean; 596 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 4.5 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2024-09-16T08:49:20.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:49:20.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:49:20.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:49:20.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:20.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-19"}]': finished 2024-09-16T08:49:20.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[40028]: from='client.56972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi160-75741-31"}]': finished 2024-09-16T08:49:20.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[40028]: from='client.56984 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi160-75757-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:20.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[40028]: osdmap e178: 8 total, 8 up, 8 in 2024-09-16T08:49:20.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:20.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:20 smithi160 ceph-mon[40028]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:20.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:20 smithi169 ceph-mon[33465]: pgmap v213: 236 pgs: 236 active+clean; 596 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 4.5 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2024-09-16T08:49:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:20 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:49:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:20 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:49:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:20 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:49:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:20 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-19"}]': finished 2024-09-16T08:49:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:20 smithi169 ceph-mon[33465]: from='client.56972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi160-75741-31"}]': finished 2024-09-16T08:49:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:20 smithi169 ceph-mon[33465]: from='client.56984 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi160-75757-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:20 smithi169 ceph-mon[33465]: osdmap e178: 8 total, 8 up, 8 in 2024-09-16T08:49:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1773788461' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:20 smithi169 ceph-mon[33465]: from='client.56972 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi160-75741-31"} : dispatch 2024-09-16T08:49:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/58716442' entity='client.admin' cmd={ 2024-09-16T08:49:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: "prefix": "osd pool set", 2024-09-16T08:49:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: "pool": "PoolEIOFlag_smithi160-75757-33", 2024-09-16T08:49:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: "var": "eio", 2024-09-16T08:49:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: "val": "true" 2024-09-16T08:49:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: } : dispatch 2024-09-16T08:49:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: from='client.56984 ' entity='client.admin' cmd={ 2024-09-16T08:49:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: "prefix": "osd pool set", 2024-09-16T08:49:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: "pool": "PoolEIOFlag_smithi160-75757-33", 2024-09-16T08:49:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: "var": "eio", 2024-09-16T08:49:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: "val": "true" 2024-09-16T08:49:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: } : dispatch 2024-09-16T08:49:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: from='client.56972 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi160-75741-31"}]': finished 2024-09-16T08:49:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: from='client.56984 ' entity='client.admin' cmd='[{ 2024-09-16T08:49:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: "prefix": "osd pool set", 2024-09-16T08:49:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: "pool": "PoolEIOFlag_smithi160-75757-33", 2024-09-16T08:49:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: "var": "eio", 2024-09-16T08:49:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: "val": "true" 2024-09-16T08:49:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: }]': finished 2024-09-16T08:49:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: osdmap e179: 8 total, 8 up, 8 in 2024-09-16T08:49:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:21.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:21.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi160-75741-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:21.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:21 smithi169 ceph-mon[33465]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi160-75741-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/58716442' entity='client.admin' cmd={ 2024-09-16T08:49:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: "prefix": "osd pool set", 2024-09-16T08:49:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: "pool": "PoolEIOFlag_smithi160-75757-33", 2024-09-16T08:49:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: "var": "eio", 2024-09-16T08:49:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: "val": "true" 2024-09-16T08:49:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: } : dispatch 2024-09-16T08:49:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: from='client.56984 ' entity='client.admin' cmd={ 2024-09-16T08:49:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: "prefix": "osd pool set", 2024-09-16T08:49:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: "pool": "PoolEIOFlag_smithi160-75757-33", 2024-09-16T08:49:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: "var": "eio", 2024-09-16T08:49:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: "val": "true" 2024-09-16T08:49:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: } : dispatch 2024-09-16T08:49:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: from='client.56972 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi160-75741-31"}]': finished 2024-09-16T08:49:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: from='client.56984 ' entity='client.admin' cmd='[{ 2024-09-16T08:49:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: "prefix": "osd pool set", 2024-09-16T08:49:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: "pool": "PoolEIOFlag_smithi160-75757-33", 2024-09-16T08:49:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: "var": "eio", 2024-09-16T08:49:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: "val": "true" 2024-09-16T08:49:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: }]': finished 2024-09-16T08:49:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: osdmap e179: 8 total, 8 up, 8 in 2024-09-16T08:49:21.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:21.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:21.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:21.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:21.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi160-75741-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:21.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[40028]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi160-75741-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:21.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/58716442' entity='client.admin' cmd={ 2024-09-16T08:49:21.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: "prefix": "osd pool set", 2024-09-16T08:49:21.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: "pool": "PoolEIOFlag_smithi160-75757-33", 2024-09-16T08:49:21.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: "var": "eio", 2024-09-16T08:49:21.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: "val": "true" 2024-09-16T08:49:21.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: } : dispatch 2024-09-16T08:49:21.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: from='client.56984 ' entity='client.admin' cmd={ 2024-09-16T08:49:21.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: "prefix": "osd pool set", 2024-09-16T08:49:21.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: "pool": "PoolEIOFlag_smithi160-75757-33", 2024-09-16T08:49:21.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: "var": "eio", 2024-09-16T08:49:21.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: "val": "true" 2024-09-16T08:49:21.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: } : dispatch 2024-09-16T08:49:21.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: from='client.56972 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi160-75741-31"}]': finished 2024-09-16T08:49:21.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: from='client.56984 ' entity='client.admin' cmd='[{ 2024-09-16T08:49:21.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: "prefix": "osd pool set", 2024-09-16T08:49:21.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: "pool": "PoolEIOFlag_smithi160-75757-33", 2024-09-16T08:49:21.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: "var": "eio", 2024-09-16T08:49:21.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: "val": "true" 2024-09-16T08:49:21.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: }]': finished 2024-09-16T08:49:21.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: osdmap e179: 8 total, 8 up, 8 in 2024-09-16T08:49:21.703 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:21.703 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:21.703 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:21.703 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:21.703 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi160-75741-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:21.703 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:21 smithi160 ceph-mon[30137]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi160-75741-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:22.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:22 smithi169 ceph-mon[33465]: pgmap v216: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T08:49:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:22 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:22 smithi169 ceph-mon[33465]: from='client.56990 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi160-75741-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:22 smithi169 ceph-mon[33465]: osdmap e180: 8 total, 8 up, 8 in 2024-09-16T08:49:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:22 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:22 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi160-75741-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:22 smithi169 ceph-mon[33465]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi160-75741-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:22.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:22 smithi160 ceph-mon[40028]: pgmap v216: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T08:49:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:22 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:22 smithi160 ceph-mon[40028]: from='client.56990 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi160-75741-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:22 smithi160 ceph-mon[40028]: osdmap e180: 8 total, 8 up, 8 in 2024-09-16T08:49:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:22 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:22 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi160-75741-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:22 smithi160 ceph-mon[40028]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi160-75741-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:22.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:22 smithi160 ceph-mon[30137]: pgmap v216: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T08:49:22.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:22 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:22.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:22 smithi160 ceph-mon[30137]: from='client.56990 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi160-75741-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:22.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:22 smithi160 ceph-mon[30137]: osdmap e180: 8 total, 8 up, 8 in 2024-09-16T08:49:22.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:22 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:22.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:22 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi160-75741-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:22.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:22 smithi160 ceph-mon[30137]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi160-75741-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:24.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:24 smithi160 ceph-mon[40028]: pgmap v219: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:24 smithi160 ceph-mon[40028]: osdmap e181: 8 total, 8 up, 8 in 2024-09-16T08:49:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2993093641' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi160-75757-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:24 smithi160 ceph-mon[40028]: from='client.56996 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi160-75757-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:24.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:24 smithi160 ceph-mon[30137]: pgmap v219: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:24 smithi160 ceph-mon[30137]: osdmap e181: 8 total, 8 up, 8 in 2024-09-16T08:49:24.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2993093641' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi160-75757-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:24.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:24 smithi160 ceph-mon[30137]: from='client.56996 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi160-75757-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:24.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:24.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:24 smithi169 ceph-mon[33465]: pgmap v219: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:24 smithi169 ceph-mon[33465]: osdmap e181: 8 total, 8 up, 8 in 2024-09-16T08:49:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2993093641' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi160-75757-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:24 smithi169 ceph-mon[33465]: from='client.56996 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi160-75757-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:25.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:49:24 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:49:25.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:25 smithi160 ceph-mon[30137]: from='client.56990 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi160-75741-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi160-75741-32"}]': finished 2024-09-16T08:49:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:25 smithi160 ceph-mon[30137]: from='client.56996 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi160-75757-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:25 smithi160 ceph-mon[30137]: osdmap e182: 8 total, 8 up, 8 in 2024-09-16T08:49:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-21"} : dispatch 2024-09-16T08:49:25.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:49:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:49:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:49:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:25 smithi160 ceph-mon[40028]: from='client.56990 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi160-75741-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi160-75741-32"}]': finished 2024-09-16T08:49:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:25 smithi160 ceph-mon[40028]: from='client.56996 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi160-75757-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:25 smithi160 ceph-mon[40028]: osdmap e182: 8 total, 8 up, 8 in 2024-09-16T08:49:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-21"} : dispatch 2024-09-16T08:49:25.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:25 smithi169 ceph-mon[33465]: from='client.56990 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi160-75741-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi160-75741-32"}]': finished 2024-09-16T08:49:25.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:25 smithi169 ceph-mon[33465]: from='client.56996 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi160-75757-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:25.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:25.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:25 smithi169 ceph-mon[33465]: osdmap e182: 8 total, 8 up, 8 in 2024-09-16T08:49:25.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-21"} : dispatch 2024-09-16T08:49:26.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:26.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:26 smithi169 ceph-mon[33465]: pgmap v222: 268 pgs: 8 unknown, 32 creating+peering, 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:49:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-21"}]': finished 2024-09-16T08:49:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:26 smithi169 ceph-mon[33465]: osdmap e183: 8 total, 8 up, 8 in 2024-09-16T08:49:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-21", "mode": "writeback"} : dispatch 2024-09-16T08:49:26.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:26.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:26 smithi160 ceph-mon[30137]: pgmap v222: 268 pgs: 8 unknown, 32 creating+peering, 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:49:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-21"}]': finished 2024-09-16T08:49:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:26 smithi160 ceph-mon[30137]: osdmap e183: 8 total, 8 up, 8 in 2024-09-16T08:49:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-21", "mode": "writeback"} : dispatch 2024-09-16T08:49:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:26 smithi160 ceph-mon[40028]: pgmap v222: 268 pgs: 8 unknown, 32 creating+peering, 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:49:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-21"}]': finished 2024-09-16T08:49:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:26 smithi160 ceph-mon[40028]: osdmap e183: 8 total, 8 up, 8 in 2024-09-16T08:49:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-21", "mode": "writeback"} : dispatch 2024-09-16T08:49:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:27 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-21", "mode": "writeback"}]': finished 2024-09-16T08:49:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:27 smithi169 ceph-mon[33465]: osdmap e184: 8 total, 8 up, 8 in 2024-09-16T08:49:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2172081546' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi160-75757-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:27 smithi169 ceph-mon[33465]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2172081546' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi160-75757-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:27 smithi169 ceph-mon[33465]: from='client.56990 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi160-75741-32"}]': finished 2024-09-16T08:49:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:27.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:27 smithi169 ceph-mon[33465]: osdmap e185: 8 total, 8 up, 8 in 2024-09-16T08:49:27.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:27.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-21"} : dispatch 2024-09-16T08:49:27.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:27 smithi169 ceph-mon[33465]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-21", "mode": "writeback"}]': finished 2024-09-16T08:49:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[30137]: osdmap e184: 8 total, 8 up, 8 in 2024-09-16T08:49:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2172081546' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi160-75757-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[30137]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2172081546' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi160-75757-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[30137]: from='client.56990 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi160-75741-32"}]': finished 2024-09-16T08:49:27.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:27.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[30137]: osdmap e185: 8 total, 8 up, 8 in 2024-09-16T08:49:27.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:27.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-21"} : dispatch 2024-09-16T08:49:27.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[30137]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:27.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:27.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-21", "mode": "writeback"}]': finished 2024-09-16T08:49:27.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[40028]: osdmap e184: 8 total, 8 up, 8 in 2024-09-16T08:49:27.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2172081546' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi160-75757-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:27.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:27.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[40028]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:27.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:27.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2172081546' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi160-75757-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:27.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[40028]: from='client.56990 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi160-75741-32"}]': finished 2024-09-16T08:49:27.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:27.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[40028]: osdmap e185: 8 total, 8 up, 8 in 2024-09-16T08:49:27.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4185711401' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:27.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-21"} : dispatch 2024-09-16T08:49:27.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:27 smithi160 ceph-mon[40028]: from='client.56990 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi160-75741-32"} : dispatch 2024-09-16T08:49:28.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:28 smithi169 ceph-mon[33465]: pgmap v225: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:49:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:28 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-21"}]': finished 2024-09-16T08:49:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:28 smithi169 ceph-mon[33465]: from='client.56990 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi160-75741-32"}]': finished 2024-09-16T08:49:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:28 smithi169 ceph-mon[33465]: osdmap e186: 8 total, 8 up, 8 in 2024-09-16T08:49:28.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:28 smithi160 ceph-mon[30137]: pgmap v225: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:49:28.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:28 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:28.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-21"}]': finished 2024-09-16T08:49:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:28 smithi160 ceph-mon[30137]: from='client.56990 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi160-75741-32"}]': finished 2024-09-16T08:49:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:28 smithi160 ceph-mon[30137]: osdmap e186: 8 total, 8 up, 8 in 2024-09-16T08:49:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:28 smithi160 ceph-mon[40028]: pgmap v225: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:49:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:28 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-21"}]': finished 2024-09-16T08:49:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:28 smithi160 ceph-mon[40028]: from='client.56990 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi160-75741-32"}]': finished 2024-09-16T08:49:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:28 smithi160 ceph-mon[40028]: osdmap e186: 8 total, 8 up, 8 in 2024-09-16T08:49:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:29 smithi160 ceph-mon[30137]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:29 smithi160 ceph-mon[30137]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi160-75741-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:29 smithi160 ceph-mon[30137]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi160-75741-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:29 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:29 smithi160 ceph-mon[40028]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:29 smithi160 ceph-mon[40028]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi160-75741-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:29 smithi160 ceph-mon[40028]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi160-75741-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:29 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:29.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:29 smithi169 ceph-mon[33465]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:29 smithi169 ceph-mon[33465]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi160-75741-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:29 smithi169 ceph-mon[33465]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi160-75741-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:29 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:30.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:30 smithi169 ceph-mon[33465]: pgmap v228: 228 pgs: 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-16T08:49:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:30 smithi169 ceph-mon[33465]: from='client.57008 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi160-75741-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:30 smithi169 ceph-mon[33465]: osdmap e187: 8 total, 8 up, 8 in 2024-09-16T08:49:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi160-75741-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:30 smithi169 ceph-mon[33465]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi160-75741-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1633867164' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-36","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:30.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:30 smithi160 ceph-mon[30137]: pgmap v228: 228 pgs: 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-16T08:49:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:30 smithi160 ceph-mon[30137]: from='client.57008 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi160-75741-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:30 smithi160 ceph-mon[30137]: osdmap e187: 8 total, 8 up, 8 in 2024-09-16T08:49:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi160-75741-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:30 smithi160 ceph-mon[30137]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi160-75741-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1633867164' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-36","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:30 smithi160 ceph-mon[40028]: pgmap v228: 228 pgs: 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-16T08:49:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:30 smithi160 ceph-mon[40028]: from='client.57008 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi160-75741-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:30 smithi160 ceph-mon[40028]: osdmap e187: 8 total, 8 up, 8 in 2024-09-16T08:49:30.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi160-75741-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:30.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:30 smithi160 ceph-mon[40028]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi160-75741-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:30.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1633867164' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-36","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:31.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1633867164' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:31 smithi169 ceph-mon[33465]: osdmap e188: 8 total, 8 up, 8 in 2024-09-16T08:49:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:31 smithi169 ceph-mon[33465]: from='client.57008 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi160-75741-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi160-75741-33"}]': finished 2024-09-16T08:49:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:31 smithi169 ceph-mon[33465]: osdmap e189: 8 total, 8 up, 8 in 2024-09-16T08:49:31.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/568890433' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:31.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1633867164' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:31.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:31 smithi160 ceph-mon[40028]: osdmap e188: 8 total, 8 up, 8 in 2024-09-16T08:49:31.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:31.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:31 smithi160 ceph-mon[40028]: from='client.57008 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi160-75741-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi160-75741-33"}]': finished 2024-09-16T08:49:31.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:31.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:31 smithi160 ceph-mon[40028]: osdmap e189: 8 total, 8 up, 8 in 2024-09-16T08:49:31.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/568890433' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1633867164' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:31 smithi160 ceph-mon[30137]: osdmap e188: 8 total, 8 up, 8 in 2024-09-16T08:49:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:31 smithi160 ceph-mon[30137]: from='client.57008 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi160-75741-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi160-75741-33"}]': finished 2024-09-16T08:49:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:31 smithi160 ceph-mon[30137]: osdmap e189: 8 total, 8 up, 8 in 2024-09-16T08:49:31.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/568890433' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:32.517 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:32 smithi169 ceph-mon[33465]: pgmap v231: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:49:32.517 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/568890433' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:32.517 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:32 smithi169 ceph-mon[33465]: osdmap e190: 8 total, 8 up, 8 in 2024-09-16T08:49:32.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:32 smithi160 ceph-mon[30137]: pgmap v231: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:49:32.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/568890433' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:32 smithi160 ceph-mon[30137]: osdmap e190: 8 total, 8 up, 8 in 2024-09-16T08:49:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:32 smithi160 ceph-mon[40028]: pgmap v231: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:49:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/568890433' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:32 smithi160 ceph-mon[40028]: osdmap e190: 8 total, 8 up, 8 in 2024-09-16T08:49:33.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:33.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:33.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:33 smithi169 ceph-mon[33465]: osdmap e191: 8 total, 8 up, 8 in 2024-09-16T08:49:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-23"} : dispatch 2024-09-16T08:49:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2426491462' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:33 smithi169 ceph-mon[33465]: from='client.57020 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:33 smithi169 ceph-mon[33465]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:33.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:33 smithi160 ceph-mon[30137]: osdmap e191: 8 total, 8 up, 8 in 2024-09-16T08:49:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-23"} : dispatch 2024-09-16T08:49:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2426491462' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:33 smithi160 ceph-mon[30137]: from='client.57020 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:33 smithi160 ceph-mon[30137]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:33.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:33 smithi160 ceph-mon[40028]: osdmap e191: 8 total, 8 up, 8 in 2024-09-16T08:49:33.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-23"} : dispatch 2024-09-16T08:49:33.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2426491462' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:33.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:33 smithi160 ceph-mon[40028]: from='client.57020 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:33.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:33.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:33 smithi160 ceph-mon[40028]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:34 smithi169 ceph-mon[33465]: pgmap v234: 300 pgs: 104 unknown, 196 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:49:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:49:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-23"}]': finished 2024-09-16T08:49:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:34 smithi169 ceph-mon[33465]: from='client.57020 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:34 smithi169 ceph-mon[33465]: from='client.57008 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi160-75741-33"}]': finished 2024-09-16T08:49:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:34 smithi169 ceph-mon[33465]: osdmap e192: 8 total, 8 up, 8 in 2024-09-16T08:49:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-23", "mode": "writeback"} : dispatch 2024-09-16T08:49:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:34 smithi169 ceph-mon[33465]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[30137]: pgmap v234: 300 pgs: 104 unknown, 196 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:49:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:49:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-23"}]': finished 2024-09-16T08:49:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[30137]: from='client.57020 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[30137]: from='client.57008 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi160-75741-33"}]': finished 2024-09-16T08:49:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[30137]: osdmap e192: 8 total, 8 up, 8 in 2024-09-16T08:49:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-23", "mode": "writeback"} : dispatch 2024-09-16T08:49:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[30137]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[40028]: pgmap v234: 300 pgs: 104 unknown, 196 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:49:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:49:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-23"}]': finished 2024-09-16T08:49:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[40028]: from='client.57020 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi160-75757-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[40028]: from='client.57008 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi160-75741-33"}]': finished 2024-09-16T08:49:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[40028]: osdmap e192: 8 total, 8 up, 8 in 2024-09-16T08:49:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2221479050' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-23", "mode": "writeback"} : dispatch 2024-09-16T08:49:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-mon[40028]: from='client.57008 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi160-75741-33"} : dispatch 2024-09-16T08:49:35.276 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:49:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:49:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:49:35.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:49:34 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:49:35.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:35 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:35.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-23", "mode": "writeback"}]': finished 2024-09-16T08:49:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:35 smithi169 ceph-mon[33465]: from='client.57008 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi160-75741-33"}]': finished 2024-09-16T08:49:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:35 smithi169 ceph-mon[33465]: osdmap e193: 8 total, 8 up, 8 in 2024-09-16T08:49:35.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:35 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-23", "mode": "writeback"}]': finished 2024-09-16T08:49:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:35 smithi160 ceph-mon[30137]: from='client.57008 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi160-75741-33"}]': finished 2024-09-16T08:49:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:35 smithi160 ceph-mon[30137]: osdmap e193: 8 total, 8 up, 8 in 2024-09-16T08:49:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:35 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-23", "mode": "writeback"}]': finished 2024-09-16T08:49:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:35 smithi160 ceph-mon[40028]: from='client.57008 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi160-75741-33"}]': finished 2024-09-16T08:49:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:35 smithi160 ceph-mon[40028]: osdmap e193: 8 total, 8 up, 8 in 2024-09-16T08:49:36.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:36 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:36 smithi169 ceph-mon[33465]: pgmap v237: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-16T08:49:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:36 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:36 smithi169 ceph-mon[33465]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:36 smithi169 ceph-mon[33465]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi160-75741-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:36 smithi169 ceph-mon[33465]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi160-75741-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:36 smithi169 ceph-mon[33465]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi160-75741-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:36 smithi169 ceph-mon[33465]: osdmap e194: 8 total, 8 up, 8 in 2024-09-16T08:49:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi160-75741-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:36 smithi169 ceph-mon[33465]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi160-75741-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[40028]: pgmap v237: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-16T08:49:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[40028]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[40028]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi160-75741-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[40028]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi160-75741-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[40028]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi160-75741-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[40028]: osdmap e194: 8 total, 8 up, 8 in 2024-09-16T08:49:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi160-75741-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[40028]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi160-75741-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:36.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[30137]: pgmap v237: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-16T08:49:36.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:36.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[30137]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[30137]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi160-75741-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:36.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[30137]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi160-75741-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:36.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[30137]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi160-75741-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:36.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[30137]: osdmap e194: 8 total, 8 up, 8 in 2024-09-16T08:49:36.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi160-75741-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:36.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:36 smithi160 ceph-mon[30137]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi160-75741-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:37.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:37.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:37 smithi169 ceph-mon[33465]: osdmap e195: 8 total, 8 up, 8 in 2024-09-16T08:49:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-23"} : dispatch 2024-09-16T08:49:37.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:37 smithi160 ceph-mon[40028]: osdmap e195: 8 total, 8 up, 8 in 2024-09-16T08:49:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-23"} : dispatch 2024-09-16T08:49:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:37 smithi160 ceph-mon[30137]: osdmap e195: 8 total, 8 up, 8 in 2024-09-16T08:49:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-23"} : dispatch 2024-09-16T08:49:38.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:38 smithi169 ceph-mon[33465]: pgmap v240: 260 pgs: 260 active+clean; 584 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-16T08:49:38.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:38 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:38 smithi169 ceph-mon[33465]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi160-75741-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi160-75741-34"}]': finished 2024-09-16T08:49:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-23"}]': finished 2024-09-16T08:49:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:38 smithi169 ceph-mon[33465]: osdmap e196: 8 total, 8 up, 8 in 2024-09-16T08:49:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1329000456' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:38 smithi169 ceph-mon[33465]: from='client.57032 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:38.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:38 smithi160 ceph-mon[40028]: pgmap v240: 260 pgs: 260 active+clean; 584 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-16T08:49:38.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:38 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:38 smithi160 ceph-mon[40028]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi160-75741-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi160-75741-34"}]': finished 2024-09-16T08:49:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-23"}]': finished 2024-09-16T08:49:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:38 smithi160 ceph-mon[40028]: osdmap e196: 8 total, 8 up, 8 in 2024-09-16T08:49:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1329000456' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:38 smithi160 ceph-mon[40028]: from='client.57032 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:38 smithi160 ceph-mon[30137]: pgmap v240: 260 pgs: 260 active+clean; 584 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-16T08:49:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:38 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:38 smithi160 ceph-mon[30137]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi160-75741-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi160-75741-34"}]': finished 2024-09-16T08:49:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-23"}]': finished 2024-09-16T08:49:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:38 smithi160 ceph-mon[30137]: osdmap e196: 8 total, 8 up, 8 in 2024-09-16T08:49:38.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1329000456' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:38.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:38 smithi160 ceph-mon[30137]: from='client.57032 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:40.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:40 smithi169 ceph-mon[33465]: pgmap v243: 268 pgs: 40 creating+peering, 228 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-16T08:49:40.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:40 smithi169 ceph-mon[33465]: from='client.57032 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:40 smithi169 ceph-mon[33465]: osdmap e197: 8 total, 8 up, 8 in 2024-09-16T08:49:40.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:40 smithi160 ceph-mon[30137]: pgmap v243: 268 pgs: 40 creating+peering, 228 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-16T08:49:40.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:40 smithi160 ceph-mon[30137]: from='client.57032 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:40 smithi160 ceph-mon[30137]: osdmap e197: 8 total, 8 up, 8 in 2024-09-16T08:49:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:40 smithi160 ceph-mon[40028]: pgmap v243: 268 pgs: 40 creating+peering, 228 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-16T08:49:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:40 smithi160 ceph-mon[40028]: from='client.57032 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:40 smithi160 ceph-mon[40028]: osdmap e197: 8 total, 8 up, 8 in 2024-09-16T08:49:41.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:41 smithi169 ceph-mon[33465]: osdmap e198: 8 total, 8 up, 8 in 2024-09-16T08:49:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3814362059' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:41 smithi169 ceph-mon[33465]: from='client.57038 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:41 smithi169 ceph-mon[33465]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:41.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:41 smithi160 ceph-mon[30137]: osdmap e198: 8 total, 8 up, 8 in 2024-09-16T08:49:41.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3814362059' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:41.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:41.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:41.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:41 smithi160 ceph-mon[30137]: from='client.57038 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:41.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:41 smithi160 ceph-mon[30137]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:41.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:41.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:41 smithi160 ceph-mon[40028]: osdmap e198: 8 total, 8 up, 8 in 2024-09-16T08:49:41.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3814362059' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:41.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:41.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:41.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:41 smithi160 ceph-mon[40028]: from='client.57038 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:41.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:41 smithi160 ceph-mon[40028]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:41.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:42.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:42 smithi169 ceph-mon[33465]: pgmap v246: 292 pgs: 64 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:49:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:42 smithi169 ceph-mon[33465]: from='client.57038 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:42 smithi169 ceph-mon[33465]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi160-75741-34"}]': finished 2024-09-16T08:49:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:42 smithi169 ceph-mon[33465]: osdmap e199: 8 total, 8 up, 8 in 2024-09-16T08:49:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:42 smithi169 ceph-mon[33465]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:42 smithi169 ceph-mon[33465]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi160-75741-34"}]': finished 2024-09-16T08:49:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:42 smithi169 ceph-mon[33465]: osdmap e200: 8 total, 8 up, 8 in 2024-09-16T08:49:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/369086116' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:42 smithi169 ceph-mon[33465]: from='client.57044 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:42.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[30137]: pgmap v246: 292 pgs: 64 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:49:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[30137]: from='client.57038 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[30137]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi160-75741-34"}]': finished 2024-09-16T08:49:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[30137]: osdmap e199: 8 total, 8 up, 8 in 2024-09-16T08:49:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[30137]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[30137]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi160-75741-34"}]': finished 2024-09-16T08:49:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[30137]: osdmap e200: 8 total, 8 up, 8 in 2024-09-16T08:49:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/369086116' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[30137]: from='client.57044 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:42.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[40028]: pgmap v246: 292 pgs: 64 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:49:42.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[40028]: from='client.57038 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:42.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[40028]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi160-75741-34"}]': finished 2024-09-16T08:49:42.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1707208195' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:42.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[40028]: osdmap e199: 8 total, 8 up, 8 in 2024-09-16T08:49:42.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[40028]: from='client.57026 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi160-75741-34"} : dispatch 2024-09-16T08:49:42.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[40028]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi160-75741-34"}]': finished 2024-09-16T08:49:42.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[40028]: osdmap e200: 8 total, 8 up, 8 in 2024-09-16T08:49:42.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/369086116' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:42.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:42 smithi160 ceph-mon[40028]: from='client.57044 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi160-75741-35"} : dispatch 2024-09-16T08:49:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi160-75741-35"} : dispatch 2024-09-16T08:49:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi160-75741-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:43 smithi169 ceph-mon[33465]: from='client.57044 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi160-75741-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:43 smithi169 ceph-mon[33465]: osdmap e201: 8 total, 8 up, 8 in 2024-09-16T08:49:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-25"} : dispatch 2024-09-16T08:49:43.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi160-75741-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi160-75741-35"} : dispatch 2024-09-16T08:49:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi160-75741-35"} : dispatch 2024-09-16T08:49:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi160-75741-35"} : dispatch 2024-09-16T08:49:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi160-75741-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[30137]: from='client.57044 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi160-75741-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[30137]: osdmap e201: 8 total, 8 up, 8 in 2024-09-16T08:49:43.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-25"} : dispatch 2024-09-16T08:49:43.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi160-75741-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi160-75741-35"} : dispatch 2024-09-16T08:49:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi160-75741-35"} : dispatch 2024-09-16T08:49:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi160-75741-35"} : dispatch 2024-09-16T08:49:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi160-75741-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[40028]: from='client.57044 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi160-75741-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[40028]: osdmap e201: 8 total, 8 up, 8 in 2024-09-16T08:49:43.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-25"} : dispatch 2024-09-16T08:49:43.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi160-75741-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi160-75741-35"} : dispatch 2024-09-16T08:49:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:44 smithi169 ceph-mon[33465]: pgmap v249: 324 pgs: 96 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:44 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-25"}]': finished 2024-09-16T08:49:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:44 smithi169 ceph-mon[33465]: osdmap e202: 8 total, 8 up, 8 in 2024-09-16T08:49:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-25", "mode": "writeback"} : dispatch 2024-09-16T08:49:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2590803293' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:44 smithi160 ceph-mon[30137]: pgmap v249: 324 pgs: 96 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:44 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-25"}]': finished 2024-09-16T08:49:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:44 smithi160 ceph-mon[30137]: osdmap e202: 8 total, 8 up, 8 in 2024-09-16T08:49:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-25", "mode": "writeback"} : dispatch 2024-09-16T08:49:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2590803293' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:44.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:44 smithi160 ceph-mon[40028]: pgmap v249: 324 pgs: 96 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:44.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:44 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:44.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-25"}]': finished 2024-09-16T08:49:44.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:44 smithi160 ceph-mon[40028]: osdmap e202: 8 total, 8 up, 8 in 2024-09-16T08:49:44.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-25", "mode": "writeback"} : dispatch 2024-09-16T08:49:44.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2590803293' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:45.291 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:49:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:49:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:49:45.291 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:49:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:49:45.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:45 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:45.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi160-75741-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi160-75741-35"}]': finished 2024-09-16T08:49:45.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-25", "mode": "writeback"}]': finished 2024-09-16T08:49:45.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2590803293' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:45.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:45 smithi169 ceph-mon[33465]: osdmap e203: 8 total, 8 up, 8 in 2024-09-16T08:49:45.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:45.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:45 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi160-75741-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi160-75741-35"}]': finished 2024-09-16T08:49:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-25", "mode": "writeback"}]': finished 2024-09-16T08:49:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2590803293' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:45 smithi160 ceph-mon[30137]: osdmap e203: 8 total, 8 up, 8 in 2024-09-16T08:49:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:45 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi160-75741-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi160-75741-35"}]': finished 2024-09-16T08:49:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-25", "mode": "writeback"}]': finished 2024-09-16T08:49:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2590803293' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:45 smithi160 ceph-mon[40028]: osdmap e203: 8 total, 8 up, 8 in 2024-09-16T08:49:45.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:46.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:46 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:46 smithi169 ceph-mon[33465]: pgmap v252: 356 pgs: 32 unknown, 324 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:49:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:46 smithi169 ceph-mon[33465]: osdmap e204: 8 total, 8 up, 8 in 2024-09-16T08:49:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-25"} : dispatch 2024-09-16T08:49:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/675468427' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:46 smithi169 ceph-mon[33465]: from='client.57062 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:46 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:46 smithi160 ceph-mon[30137]: pgmap v252: 356 pgs: 32 unknown, 324 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:49:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:46 smithi160 ceph-mon[30137]: osdmap e204: 8 total, 8 up, 8 in 2024-09-16T08:49:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-25"} : dispatch 2024-09-16T08:49:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/675468427' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:46 smithi160 ceph-mon[30137]: from='client.57062 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:46 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:46 smithi160 ceph-mon[40028]: pgmap v252: 356 pgs: 32 unknown, 324 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:49:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:46 smithi160 ceph-mon[40028]: osdmap e204: 8 total, 8 up, 8 in 2024-09-16T08:49:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-25"} : dispatch 2024-09-16T08:49:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/675468427' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:46 smithi160 ceph-mon[40028]: from='client.57062 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:47.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:47 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:47.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-25"}]': finished 2024-09-16T08:49:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:47 smithi169 ceph-mon[33465]: from='client.57062 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:47 smithi169 ceph-mon[33465]: osdmap e205: 8 total, 8 up, 8 in 2024-09-16T08:49:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi160-75741-35"} : dispatch 2024-09-16T08:49:47.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:47 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:47.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-25"}]': finished 2024-09-16T08:49:47.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:47 smithi160 ceph-mon[30137]: from='client.57062 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:47.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:47 smithi160 ceph-mon[30137]: osdmap e205: 8 total, 8 up, 8 in 2024-09-16T08:49:47.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi160-75741-35"} : dispatch 2024-09-16T08:49:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:47 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-25"}]': finished 2024-09-16T08:49:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:47 smithi160 ceph-mon[40028]: from='client.57062 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi160-75757-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:47 smithi160 ceph-mon[40028]: osdmap e205: 8 total, 8 up, 8 in 2024-09-16T08:49:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi160-75741-35"} : dispatch 2024-09-16T08:49:48.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:48 smithi169 ceph-mon[33465]: pgmap v255: 396 pgs: 72 unknown, 324 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:49:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi160-75741-35"}]': finished 2024-09-16T08:49:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:48 smithi169 ceph-mon[33465]: osdmap e206: 8 total, 8 up, 8 in 2024-09-16T08:49:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi160-75741-35"} : dispatch 2024-09-16T08:49:48.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:48 smithi160 ceph-mon[30137]: pgmap v255: 396 pgs: 72 unknown, 324 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:49:48.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi160-75741-35"}]': finished 2024-09-16T08:49:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:48 smithi160 ceph-mon[30137]: osdmap e206: 8 total, 8 up, 8 in 2024-09-16T08:49:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi160-75741-35"} : dispatch 2024-09-16T08:49:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:48 smithi160 ceph-mon[40028]: pgmap v255: 396 pgs: 72 unknown, 324 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:49:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi160-75741-35"}]': finished 2024-09-16T08:49:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:48 smithi160 ceph-mon[40028]: osdmap e206: 8 total, 8 up, 8 in 2024-09-16T08:49:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi160-75741-35"} : dispatch 2024-09-16T08:49:49.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:49:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:49 smithi169 ceph-mon[33465]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi160-75741-35"}]': finished 2024-09-16T08:49:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:49 smithi169 ceph-mon[33465]: osdmap e207: 8 total, 8 up, 8 in 2024-09-16T08:49:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:49 smithi169 ceph-mon[33465]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:49 smithi169 ceph-mon[33465]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi160-75741-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:49 smithi169 ceph-mon[33465]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi160-75741-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:49.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:49:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[30137]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi160-75741-35"}]': finished 2024-09-16T08:49:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[30137]: osdmap e207: 8 total, 8 up, 8 in 2024-09-16T08:49:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[30137]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[30137]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi160-75741-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[30137]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi160-75741-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:49:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[40028]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1582982133' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi160-75741-35"}]': finished 2024-09-16T08:49:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[40028]: osdmap e207: 8 total, 8 up, 8 in 2024-09-16T08:49:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[40028]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[40028]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi160-75741-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:49 smithi160 ceph-mon[40028]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi160-75741-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:50.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:50 smithi169 ceph-mon[33465]: pgmap v258: 324 pgs: 324 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:49:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:50 smithi169 ceph-mon[33465]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi160-75741-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:50 smithi169 ceph-mon[33465]: osdmap e208: 8 total, 8 up, 8 in 2024-09-16T08:49:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi160-75741-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:50 smithi169 ceph-mon[33465]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi160-75741-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:50.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:50 smithi160 ceph-mon[30137]: pgmap v258: 324 pgs: 324 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:49:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:50 smithi160 ceph-mon[30137]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi160-75741-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:50 smithi160 ceph-mon[30137]: osdmap e208: 8 total, 8 up, 8 in 2024-09-16T08:49:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi160-75741-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:50 smithi160 ceph-mon[30137]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi160-75741-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:50.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:50 smithi160 ceph-mon[40028]: pgmap v258: 324 pgs: 324 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:49:50.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:50.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:50 smithi160 ceph-mon[40028]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi160-75741-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:50.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:50 smithi160 ceph-mon[40028]: osdmap e208: 8 total, 8 up, 8 in 2024-09-16T08:49:50.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi160-75741-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:50 smithi160 ceph-mon[40028]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi160-75741-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:52.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:52 smithi160 ceph-mon[40028]: pgmap v261: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:49:52.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:52 smithi160 ceph-mon[40028]: osdmap e209: 8 total, 8 up, 8 in 2024-09-16T08:49:52.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:52.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:52 smithi160 ceph-mon[30137]: pgmap v261: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:49:52.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:52 smithi160 ceph-mon[30137]: osdmap e209: 8 total, 8 up, 8 in 2024-09-16T08:49:52.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:52 smithi169 ceph-mon[33465]: pgmap v261: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:49:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:52 smithi169 ceph-mon[33465]: osdmap e209: 8 total, 8 up, 8 in 2024-09-16T08:49:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:49:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:53 smithi160 ceph-mon[30137]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi160-75741-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi160-75741-36"}]': finished 2024-09-16T08:49:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:53 smithi160 ceph-mon[30137]: osdmap e210: 8 total, 8 up, 8 in 2024-09-16T08:49:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-27"} : dispatch 2024-09-16T08:49:53.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:53 smithi160 ceph-mon[40028]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi160-75741-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi160-75741-36"}]': finished 2024-09-16T08:49:53.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:53.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:53 smithi160 ceph-mon[40028]: osdmap e210: 8 total, 8 up, 8 in 2024-09-16T08:49:53.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-27"} : dispatch 2024-09-16T08:49:53.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:53 smithi169 ceph-mon[33465]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi160-75741-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi160-75741-36"}]': finished 2024-09-16T08:49:53.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:49:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:53 smithi169 ceph-mon[33465]: osdmap e210: 8 total, 8 up, 8 in 2024-09-16T08:49:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-27"} : dispatch 2024-09-16T08:49:54.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:54 smithi160 ceph-mon[30137]: pgmap v264: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:54.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-27"}]': finished 2024-09-16T08:49:54.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:54 smithi160 ceph-mon[30137]: osdmap e211: 8 total, 8 up, 8 in 2024-09-16T08:49:54.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-27", "mode": "writeback"} : dispatch 2024-09-16T08:49:54.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3643830399' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi160-75757-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:54.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:54 smithi160 ceph-mon[30137]: from='client.57074 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi160-75757-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:54.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:54 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:54.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:54 smithi160 ceph-mon[40028]: pgmap v264: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:54.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-27"}]': finished 2024-09-16T08:49:54.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:54 smithi160 ceph-mon[40028]: osdmap e211: 8 total, 8 up, 8 in 2024-09-16T08:49:54.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-27", "mode": "writeback"} : dispatch 2024-09-16T08:49:54.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3643830399' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi160-75757-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:54.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:54 smithi160 ceph-mon[40028]: from='client.57074 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi160-75757-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:54.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:54 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:54.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:54 smithi169 ceph-mon[33465]: pgmap v264: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:49:54.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-27"}]': finished 2024-09-16T08:49:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:54 smithi169 ceph-mon[33465]: osdmap e211: 8 total, 8 up, 8 in 2024-09-16T08:49:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-27", "mode": "writeback"} : dispatch 2024-09-16T08:49:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3643830399' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi160-75757-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:54 smithi169 ceph-mon[33465]: from='client.57074 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi160-75757-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:54 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:49:55.197 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:49:54 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:49:55.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:49:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:49:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:49:55.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:55 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:55.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-27", "mode": "writeback"}]': finished 2024-09-16T08:49:55.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:55 smithi160 ceph-mon[30137]: from='client.57074 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi160-75757-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:55.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:55 smithi160 ceph-mon[30137]: osdmap e212: 8 total, 8 up, 8 in 2024-09-16T08:49:55.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:55.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:55 smithi160 ceph-mon[30137]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:55.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:55 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:55.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-27", "mode": "writeback"}]': finished 2024-09-16T08:49:55.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:55 smithi160 ceph-mon[40028]: from='client.57074 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi160-75757-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:55.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:55 smithi160 ceph-mon[40028]: osdmap e212: 8 total, 8 up, 8 in 2024-09-16T08:49:55.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:55.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:55 smithi160 ceph-mon[40028]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:55.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:55 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:49:55.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-27", "mode": "writeback"}]': finished 2024-09-16T08:49:55.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:55 smithi169 ceph-mon[33465]: from='client.57074 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi160-75757-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:55.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:55 smithi169 ceph-mon[33465]: osdmap e212: 8 total, 8 up, 8 in 2024-09-16T08:49:55.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:55.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:55 smithi169 ceph-mon[33465]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:56.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:56 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:56.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:56 smithi169 ceph-mon[33465]: pgmap v267: 260 pgs: 32 creating+peering, 228 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-09-16T08:49:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:56 smithi169 ceph-mon[33465]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi160-75741-36"}]': finished 2024-09-16T08:49:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:56 smithi169 ceph-mon[33465]: osdmap e213: 8 total, 8 up, 8 in 2024-09-16T08:49:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:56 smithi169 ceph-mon[33465]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:56 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:56 smithi160 ceph-mon[30137]: pgmap v267: 260 pgs: 32 creating+peering, 228 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-09-16T08:49:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:56 smithi160 ceph-mon[30137]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi160-75741-36"}]': finished 2024-09-16T08:49:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:56 smithi160 ceph-mon[30137]: osdmap e213: 8 total, 8 up, 8 in 2024-09-16T08:49:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:56 smithi160 ceph-mon[30137]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:56 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:49:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:56 smithi160 ceph-mon[40028]: pgmap v267: 260 pgs: 32 creating+peering, 228 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-09-16T08:49:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:56 smithi160 ceph-mon[40028]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi160-75741-36"}]': finished 2024-09-16T08:49:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:56 smithi160 ceph-mon[40028]: osdmap e213: 8 total, 8 up, 8 in 2024-09-16T08:49:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2954221370' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:56 smithi160 ceph-mon[40028]: from='client.57068 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi160-75741-36"} : dispatch 2024-09-16T08:49:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:49:57.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi160-75741-36"}]': finished 2024-09-16T08:49:57.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:57.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: osdmap e214: 8 total, 8 up, 8 in 2024-09-16T08:49:57.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1877007547' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi160-75757-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:57.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-27"} : dispatch 2024-09-16T08:49:57.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: from='client.57080 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi160-75757-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:57.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:57.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:57.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:57.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:57.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:57.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:57.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:57.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-27"}]': finished 2024-09-16T08:49:57.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: from='client.57080 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi160-75757-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:57.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:57.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:57 smithi169 ceph-mon[33465]: osdmap e215: 8 total, 8 up, 8 in 2024-09-16T08:49:57.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi160-75741-36"}]': finished 2024-09-16T08:49:57.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:57.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: osdmap e214: 8 total, 8 up, 8 in 2024-09-16T08:49:57.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1877007547' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi160-75757-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:57.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-27"} : dispatch 2024-09-16T08:49:57.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: from='client.57080 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi160-75757-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:57.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:57.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:57.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:57.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:57.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:57.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-27"}]': finished 2024-09-16T08:49:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: from='client.57080 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi160-75757-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[40028]: osdmap e215: 8 total, 8 up, 8 in 2024-09-16T08:49:57.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi160-75741-36"}]': finished 2024-09-16T08:49:57.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:49:57.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: osdmap e214: 8 total, 8 up, 8 in 2024-09-16T08:49:57.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1877007547' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi160-75757-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:57.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-27"} : dispatch 2024-09-16T08:49:57.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: from='client.57080 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi160-75757-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:57.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:57.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:57.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:57.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:57.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:57.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:57.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:49:57.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-27"}]': finished 2024-09-16T08:49:57.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: from='client.57080 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi160-75757-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:49:57.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:57.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:57 smithi160 ceph-mon[30137]: osdmap e215: 8 total, 8 up, 8 in 2024-09-16T08:49:58.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:58 smithi169 ceph-mon[33465]: pgmap v270: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-16T08:49:58.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi160-75741-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:58.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:58 smithi169 ceph-mon[33465]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi160-75741-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:58.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:58 smithi169 ceph-mon[33465]: osdmap e216: 8 total, 8 up, 8 in 2024-09-16T08:49:58.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:58 smithi160 ceph-mon[30137]: pgmap v270: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-16T08:49:58.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi160-75741-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:58.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:58 smithi160 ceph-mon[30137]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi160-75741-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:58.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:58 smithi160 ceph-mon[30137]: osdmap e216: 8 total, 8 up, 8 in 2024-09-16T08:49:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:58 smithi160 ceph-mon[40028]: pgmap v270: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-16T08:49:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi160-75741-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:58 smithi160 ceph-mon[40028]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi160-75741-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:49:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:58 smithi160 ceph-mon[40028]: osdmap e216: 8 total, 8 up, 8 in 2024-09-16T08:49:59.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-46"} : dispatch 2024-09-16T08:49:59.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-46"} : dispatch 2024-09-16T08:49:59.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi160-75757-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:59.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:59 smithi160 ceph-mon[30137]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi160-75741-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi160-75741-37"}]': finished 2024-09-16T08:49:59.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi160-75757-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:59.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:59 smithi160 ceph-mon[30137]: osdmap e217: 8 total, 8 up, 8 in 2024-09-16T08:49:59.600 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi160-75757-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi160-75757-46"} : dispatch 2024-09-16T08:49:59.600 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:49:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:59.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-46"} : dispatch 2024-09-16T08:49:59.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-46"} : dispatch 2024-09-16T08:49:59.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi160-75757-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:59.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:59 smithi160 ceph-mon[40028]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi160-75741-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi160-75741-37"}]': finished 2024-09-16T08:49:59.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi160-75757-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:59.601 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:59 smithi160 ceph-mon[40028]: osdmap e217: 8 total, 8 up, 8 in 2024-09-16T08:49:59.601 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi160-75757-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi160-75757-46"} : dispatch 2024-09-16T08:49:59.601 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:49:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:49:59.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-46"} : dispatch 2024-09-16T08:49:59.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-46"} : dispatch 2024-09-16T08:49:59.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi160-75757-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:49:59.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:59 smithi169 ceph-mon[33465]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi160-75741-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi160-75741-37"}]': finished 2024-09-16T08:49:59.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi160-75757-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:49:59.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:59 smithi169 ceph-mon[33465]: osdmap e217: 8 total, 8 up, 8 in 2024-09-16T08:49:59.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi160-75757-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi160-75757-46"} : dispatch 2024-09-16T08:49:59.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:49:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:00.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:00 smithi169 ceph-mon[33465]: pgmap v273: 196 pgs: 196 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:00.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:00 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:00.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:00 smithi169 ceph-mon[33465]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-16T08:50:00.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:00 smithi169 ceph-mon[33465]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-16T08:50:00.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:00 smithi169 ceph-mon[33465]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-16T08:50:00.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:00 smithi169 ceph-mon[33465]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-16T08:50:00.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:00.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:00 smithi169 ceph-mon[33465]: osdmap e218: 8 total, 8 up, 8 in 2024-09-16T08:50:00.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:00 smithi160 ceph-mon[30137]: pgmap v273: 196 pgs: 196 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:00 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:00 smithi160 ceph-mon[30137]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-16T08:50:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:00 smithi160 ceph-mon[30137]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-16T08:50:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:00 smithi160 ceph-mon[30137]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-16T08:50:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:00 smithi160 ceph-mon[30137]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-16T08:50:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:00 smithi160 ceph-mon[30137]: osdmap e218: 8 total, 8 up, 8 in 2024-09-16T08:50:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:00 smithi160 ceph-mon[40028]: pgmap v273: 196 pgs: 196 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:00 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:00 smithi160 ceph-mon[40028]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-16T08:50:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:00 smithi160 ceph-mon[40028]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-16T08:50:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:00 smithi160 ceph-mon[40028]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-16T08:50:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:00 smithi160 ceph-mon[40028]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-16T08:50:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:00 smithi160 ceph-mon[40028]: osdmap e218: 8 total, 8 up, 8 in 2024-09-16T08:50:02.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:02 smithi169 ceph-mon[33465]: pgmap v276: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:02.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:02 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi160-75757-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi160-75757-46"}]': finished 2024-09-16T08:50:02.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:02 smithi169 ceph-mon[33465]: osdmap e219: 8 total, 8 up, 8 in 2024-09-16T08:50:02.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:02 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:50:02.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:02 smithi169 ceph-mon[33465]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:50:02.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:02 smithi160 ceph-mon[30137]: pgmap v276: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:02.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:02 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi160-75757-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi160-75757-46"}]': finished 2024-09-16T08:50:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:02 smithi160 ceph-mon[30137]: osdmap e219: 8 total, 8 up, 8 in 2024-09-16T08:50:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:02 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:50:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:02 smithi160 ceph-mon[30137]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:50:02.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:02 smithi160 ceph-mon[40028]: pgmap v276: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:02.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:02 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi160-75757-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi160-75757-46"}]': finished 2024-09-16T08:50:02.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:02 smithi160 ceph-mon[40028]: osdmap e219: 8 total, 8 up, 8 in 2024-09-16T08:50:02.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:02 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:50:02.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:02 smithi160 ceph-mon[40028]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:50:03.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:03 smithi169 ceph-mon[33465]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37"}]': finished 2024-09-16T08:50:03.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:03 smithi169 ceph-mon[33465]: osdmap e220: 8 total, 8 up, 8 in 2024-09-16T08:50:03.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:50:03.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:03 smithi169 ceph-mon[33465]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:50:03.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:03.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:03 smithi160 ceph-mon[30137]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37"}]': finished 2024-09-16T08:50:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:03 smithi160 ceph-mon[30137]: osdmap e220: 8 total, 8 up, 8 in 2024-09-16T08:50:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:50:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:03 smithi160 ceph-mon[30137]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:50:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:03 smithi160 ceph-mon[40028]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi160-75741-37"}]': finished 2024-09-16T08:50:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:03 smithi160 ceph-mon[40028]: osdmap e220: 8 total, 8 up, 8 in 2024-09-16T08:50:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3944514368' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:50:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:03 smithi160 ceph-mon[40028]: from='client.57086 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi160-75741-37"} : dispatch 2024-09-16T08:50:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: pgmap v279: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi160-75741-37"}]': finished 2024-09-16T08:50:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: osdmap e221: 8 total, 8 up, 8 in 2024-09-16T08:50:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-29"} : dispatch 2024-09-16T08:50:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-46"} : dispatch 2024-09-16T08:50:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi160-75741-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi160-75741-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:50:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-29"}]': finished 2024-09-16T08:50:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-46"}]': finished 2024-09-16T08:50:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi160-75741-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: osdmap e222: 8 total, 8 up, 8 in 2024-09-16T08:50:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-46"} : dispatch 2024-09-16T08:50:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-29", "mode": "writeback"} : dispatch 2024-09-16T08:50:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi160-75741-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-mon[33465]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi160-75741-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: pgmap v279: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi160-75741-37"}]': finished 2024-09-16T08:50:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: osdmap e221: 8 total, 8 up, 8 in 2024-09-16T08:50:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-29"} : dispatch 2024-09-16T08:50:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-46"} : dispatch 2024-09-16T08:50:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi160-75741-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi160-75741-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:50:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-29"}]': finished 2024-09-16T08:50:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-46"}]': finished 2024-09-16T08:50:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi160-75741-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: osdmap e222: 8 total, 8 up, 8 in 2024-09-16T08:50:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-46"} : dispatch 2024-09-16T08:50:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-29", "mode": "writeback"} : dispatch 2024-09-16T08:50:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi160-75741-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[30137]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi160-75741-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: pgmap v279: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:04.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi160-75741-37"}]': finished 2024-09-16T08:50:04.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:04.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: osdmap e221: 8 total, 8 up, 8 in 2024-09-16T08:50:04.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-29"} : dispatch 2024-09-16T08:50:04.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-46"} : dispatch 2024-09-16T08:50:04.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi160-75741-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:04.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi160-75741-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:04.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:50:04.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-29"}]': finished 2024-09-16T08:50:04.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-46"}]': finished 2024-09-16T08:50:04.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi160-75741-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:04.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: osdmap e222: 8 total, 8 up, 8 in 2024-09-16T08:50:04.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-46"} : dispatch 2024-09-16T08:50:04.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-29", "mode": "writeback"} : dispatch 2024-09-16T08:50:04.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi160-75741-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:04.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-mon[40028]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi160-75741-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:05.239 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:05 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:05.239 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-46"}]': finished 2024-09-16T08:50:05.239 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-29", "mode": "writeback"}]': finished 2024-09-16T08:50:05.239 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:50:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:50:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:50:05.241 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:50:04 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:50:05.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:05 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:05.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-46"}]': finished 2024-09-16T08:50:05.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-29", "mode": "writeback"}]': finished 2024-09-16T08:50:05.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:05 smithi169 ceph-mon[33465]: osdmap e223: 8 total, 8 up, 8 in 2024-09-16T08:50:05.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-47"} : dispatch 2024-09-16T08:50:05.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-47"} : dispatch 2024-09-16T08:50:05.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi160-75757-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:05.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:05 smithi160 ceph-mon[30137]: osdmap e223: 8 total, 8 up, 8 in 2024-09-16T08:50:05.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-47"} : dispatch 2024-09-16T08:50:05.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-47"} : dispatch 2024-09-16T08:50:05.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi160-75757-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:05.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:05 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:05.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1215319506' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-46"}]': finished 2024-09-16T08:50:05.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-29", "mode": "writeback"}]': finished 2024-09-16T08:50:05.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:05 smithi160 ceph-mon[40028]: osdmap e223: 8 total, 8 up, 8 in 2024-09-16T08:50:05.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-47"} : dispatch 2024-09-16T08:50:05.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-47"} : dispatch 2024-09-16T08:50:05.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi160-75757-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:06.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:06.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:06 smithi169 ceph-mon[33465]: pgmap v282: 228 pgs: 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-16T08:50:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:06 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:06 smithi169 ceph-mon[33465]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi160-75741-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi160-75741-38"}]': finished 2024-09-16T08:50:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:06 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi160-75757-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:06 smithi169 ceph-mon[33465]: osdmap e224: 8 total, 8 up, 8 in 2024-09-16T08:50:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:06 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi160-75757-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi160-75757-47"} : dispatch 2024-09-16T08:50:06.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:06 smithi160 ceph-mon[30137]: pgmap v282: 228 pgs: 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-16T08:50:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:06 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:06 smithi160 ceph-mon[30137]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi160-75741-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi160-75741-38"}]': finished 2024-09-16T08:50:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:06 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi160-75757-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:06 smithi160 ceph-mon[30137]: osdmap e224: 8 total, 8 up, 8 in 2024-09-16T08:50:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:06 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi160-75757-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi160-75757-47"} : dispatch 2024-09-16T08:50:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:06 smithi160 ceph-mon[40028]: pgmap v282: 228 pgs: 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-16T08:50:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:06 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:06 smithi160 ceph-mon[40028]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi160-75741-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi160-75741-38"}]': finished 2024-09-16T08:50:06.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:06 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi160-75757-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:06.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:06 smithi160 ceph-mon[40028]: osdmap e224: 8 total, 8 up, 8 in 2024-09-16T08:50:06.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:06 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi160-75757-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi160-75757-47"} : dispatch 2024-09-16T08:50:08.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:08 smithi160 ceph-mon[30137]: pgmap v285: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-16T08:50:08.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:08 smithi160 ceph-mon[30137]: osdmap e225: 8 total, 8 up, 8 in 2024-09-16T08:50:08.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:08 smithi160 ceph-mon[40028]: pgmap v285: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-16T08:50:08.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:08 smithi160 ceph-mon[40028]: osdmap e225: 8 total, 8 up, 8 in 2024-09-16T08:50:08.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:08 smithi169 ceph-mon[33465]: pgmap v285: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-16T08:50:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:08 smithi169 ceph-mon[33465]: osdmap e225: 8 total, 8 up, 8 in 2024-09-16T08:50:09.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi160-75757-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi160-75757-47"}]': finished 2024-09-16T08:50:09.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:09 smithi160 ceph-mon[30137]: osdmap e226: 8 total, 8 up, 8 in 2024-09-16T08:50:09.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:09.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:09 smithi160 ceph-mon[30137]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:09.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi160-75757-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi160-75757-47"}]': finished 2024-09-16T08:50:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:09 smithi160 ceph-mon[40028]: osdmap e226: 8 total, 8 up, 8 in 2024-09-16T08:50:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:09 smithi160 ceph-mon[40028]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:09.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:09.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi160-75757-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi160-75757-47"}]': finished 2024-09-16T08:50:09.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:09 smithi169 ceph-mon[33465]: osdmap e226: 8 total, 8 up, 8 in 2024-09-16T08:50:09.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:09.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:09 smithi169 ceph-mon[33465]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:09.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:10 smithi160 ceph-mon[30137]: pgmap v288: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-09-16T08:50:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:10 smithi160 ceph-mon[30137]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi160-75741-38"}]': finished 2024-09-16T08:50:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:10 smithi160 ceph-mon[30137]: osdmap e227: 8 total, 8 up, 8 in 2024-09-16T08:50:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-29"} : dispatch 2024-09-16T08:50:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:10 smithi160 ceph-mon[30137]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:10.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:10 smithi160 ceph-mon[40028]: pgmap v288: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-09-16T08:50:10.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:10 smithi160 ceph-mon[40028]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi160-75741-38"}]': finished 2024-09-16T08:50:10.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:10.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:10 smithi160 ceph-mon[40028]: osdmap e227: 8 total, 8 up, 8 in 2024-09-16T08:50:10.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-29"} : dispatch 2024-09-16T08:50:10.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:10.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:10 smithi160 ceph-mon[40028]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:10.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:10 smithi169 ceph-mon[33465]: pgmap v288: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-09-16T08:50:10.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:10 smithi169 ceph-mon[33465]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi160-75741-38"}]': finished 2024-09-16T08:50:10.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:10 smithi169 ceph-mon[33465]: osdmap e227: 8 total, 8 up, 8 in 2024-09-16T08:50:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-29"} : dispatch 2024-09-16T08:50:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/490872404' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:10 smithi169 ceph-mon[33465]: from='client.57098 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi160-75741-38"} : dispatch 2024-09-16T08:50:11.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:11.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-29"}]': finished 2024-09-16T08:50:11.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi160-75741-38"}]': finished 2024-09-16T08:50:11.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: osdmap e228: 8 total, 8 up, 8 in 2024-09-16T08:50:11.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-47"} : dispatch 2024-09-16T08:50:11.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi160-75741-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:11.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi160-75741-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:11.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-47"}]': finished 2024-09-16T08:50:11.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi160-75741-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:11.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: osdmap e229: 8 total, 8 up, 8 in 2024-09-16T08:50:11.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi160-75741-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-47"} : dispatch 2024-09-16T08:50:11.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[30137]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi160-75741-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:11.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-29"}]': finished 2024-09-16T08:50:11.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi160-75741-38"}]': finished 2024-09-16T08:50:11.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: osdmap e228: 8 total, 8 up, 8 in 2024-09-16T08:50:11.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-47"} : dispatch 2024-09-16T08:50:11.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi160-75741-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:11.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi160-75741-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:11.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-47"}]': finished 2024-09-16T08:50:11.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi160-75741-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:11.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: osdmap e229: 8 total, 8 up, 8 in 2024-09-16T08:50:11.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi160-75741-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-47"} : dispatch 2024-09-16T08:50:11.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:11 smithi160 ceph-mon[40028]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi160-75741-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-29"}]': finished 2024-09-16T08:50:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi160-75741-38"}]': finished 2024-09-16T08:50:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: osdmap e228: 8 total, 8 up, 8 in 2024-09-16T08:50:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-47"} : dispatch 2024-09-16T08:50:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi160-75741-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi160-75741-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:11.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi160-75757-47"}]': finished 2024-09-16T08:50:11.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi160-75741-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:11.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: osdmap e229: 8 total, 8 up, 8 in 2024-09-16T08:50:11.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi160-75741-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:11.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-47"} : dispatch 2024-09-16T08:50:11.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:11 smithi169 ceph-mon[33465]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi160-75741-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:12.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:12 smithi160 ceph-mon[30137]: pgmap v291: 228 pgs: 228 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-09-16T08:50:12.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:12 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:12.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-47"}]': finished 2024-09-16T08:50:12.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:12 smithi160 ceph-mon[30137]: osdmap e230: 8 total, 8 up, 8 in 2024-09-16T08:50:12.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:12 smithi160 ceph-mon[40028]: pgmap v291: 228 pgs: 228 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-09-16T08:50:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:12 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-47"}]': finished 2024-09-16T08:50:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:12 smithi160 ceph-mon[40028]: osdmap e230: 8 total, 8 up, 8 in 2024-09-16T08:50:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:12.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:12 smithi169 ceph-mon[33465]: pgmap v291: 228 pgs: 228 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-09-16T08:50:12.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:12 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:12.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2731700047' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi160-75757-47"}]': finished 2024-09-16T08:50:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:12 smithi169 ceph-mon[33465]: osdmap e230: 8 total, 8 up, 8 in 2024-09-16T08:50:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:13.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:13 smithi169 ceph-mon[33465]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:13 smithi169 ceph-mon[33465]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi160-75757-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:13 smithi169 ceph-mon[33465]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi160-75757-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:13 smithi169 ceph-mon[33465]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi160-75741-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi160-75741-39"}]': finished 2024-09-16T08:50:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:13 smithi169 ceph-mon[33465]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi160-75757-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:13 smithi169 ceph-mon[33465]: osdmap e231: 8 total, 8 up, 8 in 2024-09-16T08:50:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi160-75757-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:13.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:13 smithi169 ceph-mon[33465]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi160-75757-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[30137]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[30137]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi160-75757-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:13.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[30137]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi160-75757-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:13.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[30137]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi160-75741-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi160-75741-39"}]': finished 2024-09-16T08:50:13.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:13.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[30137]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi160-75757-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:13.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[30137]: osdmap e231: 8 total, 8 up, 8 in 2024-09-16T08:50:13.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi160-75757-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:13.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[30137]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi160-75757-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[40028]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[40028]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi160-75757-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:13.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[40028]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi160-75757-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:13.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[40028]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi160-75741-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi160-75741-39"}]': finished 2024-09-16T08:50:13.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:13.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[40028]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi160-75757-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:13.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[40028]: osdmap e231: 8 total, 8 up, 8 in 2024-09-16T08:50:13.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi160-75757-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:13.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:13.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:13 smithi160 ceph-mon[40028]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi160-75757-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:14 smithi169 ceph-mon[33465]: pgmap v294: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:14 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:14 smithi169 ceph-mon[33465]: osdmap e232: 8 total, 8 up, 8 in 2024-09-16T08:50:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:14 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-31"} : dispatch 2024-09-16T08:50:14.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:14 smithi160 ceph-mon[40028]: pgmap v294: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:14.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:14 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:14.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:14 smithi160 ceph-mon[40028]: osdmap e232: 8 total, 8 up, 8 in 2024-09-16T08:50:14.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:14 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-31"} : dispatch 2024-09-16T08:50:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:14 smithi160 ceph-mon[30137]: pgmap v294: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:14 smithi160 ceph-mon[30137]: osdmap e232: 8 total, 8 up, 8 in 2024-09-16T08:50:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-31"} : dispatch 2024-09-16T08:50:15.349 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:50:14 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:50:15.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:50:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:50:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:50:16.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:16 smithi160 ceph-mon[30137]: pgmap v297: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:16 smithi160 ceph-mon[30137]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi160-75757-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi160-75757-48"}]': finished 2024-09-16T08:50:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-31"}]': finished 2024-09-16T08:50:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:16 smithi160 ceph-mon[30137]: osdmap e233: 8 total, 8 up, 8 in 2024-09-16T08:50:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-31", "mode": "writeback"} : dispatch 2024-09-16T08:50:16.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:16 smithi160 ceph-mon[30137]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:16.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:16.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:16 smithi160 ceph-mon[40028]: pgmap v297: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:16.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:16 smithi160 ceph-mon[40028]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi160-75757-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi160-75757-48"}]': finished 2024-09-16T08:50:16.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-31"}]': finished 2024-09-16T08:50:16.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:16 smithi160 ceph-mon[40028]: osdmap e233: 8 total, 8 up, 8 in 2024-09-16T08:50:16.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:16.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-31", "mode": "writeback"} : dispatch 2024-09-16T08:50:16.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:16 smithi160 ceph-mon[40028]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:16.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:16.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:16 smithi169 ceph-mon[33465]: pgmap v297: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:16 smithi169 ceph-mon[33465]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi160-75757-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi160-75757-48"}]': finished 2024-09-16T08:50:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-31"}]': finished 2024-09-16T08:50:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:16 smithi169 ceph-mon[33465]: osdmap e233: 8 total, 8 up, 8 in 2024-09-16T08:50:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-31", "mode": "writeback"} : dispatch 2024-09-16T08:50:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:16 smithi169 ceph-mon[33465]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-31", "mode": "writeback"}]': finished 2024-09-16T08:50:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[30137]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi160-75741-39"}]': finished 2024-09-16T08:50:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[30137]: osdmap e234: 8 total, 8 up, 8 in 2024-09-16T08:50:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[30137]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[30137]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi160-75741-39"}]': finished 2024-09-16T08:50:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:17.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[30137]: osdmap e235: 8 total, 8 up, 8 in 2024-09-16T08:50:17.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-31"} : dispatch 2024-09-16T08:50:17.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:17.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[30137]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:17.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:17.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-31", "mode": "writeback"}]': finished 2024-09-16T08:50:17.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[40028]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi160-75741-39"}]': finished 2024-09-16T08:50:17.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[40028]: osdmap e234: 8 total, 8 up, 8 in 2024-09-16T08:50:17.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:17.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[40028]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:17.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:17.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[40028]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi160-75741-39"}]': finished 2024-09-16T08:50:17.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:17.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[40028]: osdmap e235: 8 total, 8 up, 8 in 2024-09-16T08:50:17.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-31"} : dispatch 2024-09-16T08:50:17.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:17.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:17 smithi160 ceph-mon[40028]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:17.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:17 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-31", "mode": "writeback"}]': finished 2024-09-16T08:50:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:17 smithi169 ceph-mon[33465]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi160-75741-39"}]': finished 2024-09-16T08:50:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:17 smithi169 ceph-mon[33465]: osdmap e234: 8 total, 8 up, 8 in 2024-09-16T08:50:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2373667441' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:17 smithi169 ceph-mon[33465]: from='client.57110 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi160-75741-39"} : dispatch 2024-09-16T08:50:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:17 smithi169 ceph-mon[33465]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi160-75741-39"}]': finished 2024-09-16T08:50:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:17 smithi169 ceph-mon[33465]: osdmap e235: 8 total, 8 up, 8 in 2024-09-16T08:50:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-31"} : dispatch 2024-09-16T08:50:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:17 smithi169 ceph-mon[33465]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:18.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[30137]: pgmap v300: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:18.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[30137]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[30137]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi160-75741-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:18.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[30137]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi160-75741-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:18.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:18.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-31"}]': finished 2024-09-16T08:50:18.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[30137]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi160-75757-48"}]': finished 2024-09-16T08:50:18.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[30137]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi160-75741-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:18.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[30137]: osdmap e236: 8 total, 8 up, 8 in 2024-09-16T08:50:18.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi160-75741-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:18.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[30137]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi160-75741-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[30137]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:18.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[40028]: pgmap v300: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:18.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[40028]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[40028]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi160-75741-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:18.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[40028]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi160-75741-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:18.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:18.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-31"}]': finished 2024-09-16T08:50:18.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[40028]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi160-75757-48"}]': finished 2024-09-16T08:50:18.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[40028]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi160-75741-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:18.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[40028]: osdmap e236: 8 total, 8 up, 8 in 2024-09-16T08:50:18.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi160-75741-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:18.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[40028]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi160-75741-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:18 smithi160 ceph-mon[40028]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:18 smithi169 ceph-mon[33465]: pgmap v300: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:18 smithi169 ceph-mon[33465]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:18 smithi169 ceph-mon[33465]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi160-75741-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:18 smithi169 ceph-mon[33465]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi160-75741-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:18 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-31"}]': finished 2024-09-16T08:50:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:18 smithi169 ceph-mon[33465]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi160-75757-48"}]': finished 2024-09-16T08:50:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:18 smithi169 ceph-mon[33465]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi160-75741-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:18 smithi169 ceph-mon[33465]: osdmap e236: 8 total, 8 up, 8 in 2024-09-16T08:50:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi160-75741-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2319667134' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:18.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:18 smithi169 ceph-mon[33465]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi160-75741-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:18.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:18 smithi169 ceph-mon[33465]: from='client.57116 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi160-75757-48"} : dispatch 2024-09-16T08:50:19.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:50:19.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:19 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:19.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:19 smithi160 ceph-mon[30137]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi160-75757-48"}]': finished 2024-09-16T08:50:19.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:50:19.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:19 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:19.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:19 smithi160 ceph-mon[40028]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi160-75757-48"}]': finished 2024-09-16T08:50:19.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:50:19.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:19 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:19.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:19 smithi169 ceph-mon[33465]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi160-75757-48"}]': finished 2024-09-16T08:50:20.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: pgmap v303: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T08:50:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: osdmap e237: 8 total, 8 up, 8 in 2024-09-16T08:50:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi160-75757-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi160-75757-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:50:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:50:20.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:50:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:50:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi160-75741-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi160-75741-40"}]': finished 2024-09-16T08:50:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: from='client.57125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi160-75757-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: osdmap e238: 8 total, 8 up, 8 in 2024-09-16T08:50:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi160-75757-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:20.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[30137]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi160-75757-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: pgmap v303: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T08:50:20.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: osdmap e237: 8 total, 8 up, 8 in 2024-09-16T08:50:20.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi160-75757-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi160-75757-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:50:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:50:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:50:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:50:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi160-75741-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi160-75741-40"}]': finished 2024-09-16T08:50:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: from='client.57125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi160-75757-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: osdmap e238: 8 total, 8 up, 8 in 2024-09-16T08:50:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi160-75757-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:20.451 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:20 smithi160 ceph-mon[40028]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi160-75757-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: pgmap v303: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T08:50:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: osdmap e237: 8 total, 8 up, 8 in 2024-09-16T08:50:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi160-75757-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi160-75757-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:50:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:50:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:50:20.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:50:20.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi160-75741-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi160-75741-40"}]': finished 2024-09-16T08:50:20.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: from='client.57125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi160-75757-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:20.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: osdmap e238: 8 total, 8 up, 8 in 2024-09-16T08:50:20.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi160-75757-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:20.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:20.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:20 smithi169 ceph-mon[33465]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi160-75757-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:22.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:22 smithi160 ceph-mon[30137]: pgmap v306: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:50:22.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:22 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:22.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:22 smithi160 ceph-mon[30137]: osdmap e239: 8 total, 8 up, 8 in 2024-09-16T08:50:22.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:22 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:22.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:22 smithi160 ceph-mon[40028]: pgmap v306: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:50:22.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:22 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:22.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:22 smithi160 ceph-mon[40028]: osdmap e239: 8 total, 8 up, 8 in 2024-09-16T08:50:22.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:22 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:22.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:22 smithi169 ceph-mon[33465]: pgmap v306: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:50:22.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:22 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:22.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:22 smithi169 ceph-mon[33465]: osdmap e239: 8 total, 8 up, 8 in 2024-09-16T08:50:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:22 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:23.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:23 smithi160 ceph-mon[30137]: from='client.57125 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi160-75757-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi160-75757-49"}]': finished 2024-09-16T08:50:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:23 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:23 smithi160 ceph-mon[30137]: osdmap e240: 8 total, 8 up, 8 in 2024-09-16T08:50:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:23 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-33"} : dispatch 2024-09-16T08:50:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:23 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:23.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:23 smithi160 ceph-mon[30137]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:23.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:23 smithi160 ceph-mon[40028]: from='client.57125 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi160-75757-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi160-75757-49"}]': finished 2024-09-16T08:50:23.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:23 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:23.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:23 smithi160 ceph-mon[40028]: osdmap e240: 8 total, 8 up, 8 in 2024-09-16T08:50:23.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:23 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-33"} : dispatch 2024-09-16T08:50:23.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:23 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:23.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:23 smithi160 ceph-mon[40028]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:23 smithi169 ceph-mon[33465]: from='client.57125 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi160-75757-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi160-75757-49"}]': finished 2024-09-16T08:50:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:23 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:23 smithi169 ceph-mon[33465]: osdmap e240: 8 total, 8 up, 8 in 2024-09-16T08:50:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:23 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-33"} : dispatch 2024-09-16T08:50:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:23 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:23 smithi169 ceph-mon[33465]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:24.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:24 smithi160 ceph-mon[30137]: pgmap v309: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-33"}]': finished 2024-09-16T08:50:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:24 smithi160 ceph-mon[30137]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi160-75741-40"}]': finished 2024-09-16T08:50:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:24 smithi160 ceph-mon[30137]: osdmap e241: 8 total, 8 up, 8 in 2024-09-16T08:50:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-33", "mode": "writeback"} : dispatch 2024-09-16T08:50:24.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:24 smithi160 ceph-mon[30137]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:24.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:24 smithi160 ceph-mon[40028]: pgmap v309: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:24.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-33"}]': finished 2024-09-16T08:50:24.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:24 smithi160 ceph-mon[40028]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi160-75741-40"}]': finished 2024-09-16T08:50:24.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:24 smithi160 ceph-mon[40028]: osdmap e241: 8 total, 8 up, 8 in 2024-09-16T08:50:24.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:24.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-33", "mode": "writeback"} : dispatch 2024-09-16T08:50:24.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:24 smithi160 ceph-mon[40028]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:24 smithi169 ceph-mon[33465]: pgmap v309: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-33"}]': finished 2024-09-16T08:50:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:24 smithi169 ceph-mon[33465]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi160-75741-40"}]': finished 2024-09-16T08:50:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:24 smithi169 ceph-mon[33465]: osdmap e241: 8 total, 8 up, 8 in 2024-09-16T08:50:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4049925068' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:24.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-33", "mode": "writeback"} : dispatch 2024-09-16T08:50:24.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:24 smithi169 ceph-mon[33465]: from='client.57122 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi160-75741-40"} : dispatch 2024-09-16T08:50:25.349 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:50:24 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:50:25.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:25 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:25.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-33", "mode": "writeback"}]': finished 2024-09-16T08:50:25.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:25 smithi169 ceph-mon[33465]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi160-75741-40"}]': finished 2024-09-16T08:50:25.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:25 smithi169 ceph-mon[33465]: osdmap e242: 8 total, 8 up, 8 in 2024-09-16T08:50:25.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:25.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:25 smithi169 ceph-mon[33465]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:25.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi160-75741-41"} : dispatch 2024-09-16T08:50:25.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi160-75741-41"} : dispatch 2024-09-16T08:50:25.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi160-75741-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:25.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:25.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:50:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:50:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:50:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-33", "mode": "writeback"}]': finished 2024-09-16T08:50:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[30137]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi160-75741-40"}]': finished 2024-09-16T08:50:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[30137]: osdmap e242: 8 total, 8 up, 8 in 2024-09-16T08:50:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[30137]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi160-75741-41"} : dispatch 2024-09-16T08:50:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi160-75741-41"} : dispatch 2024-09-16T08:50:25.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi160-75741-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:25.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-33", "mode": "writeback"}]': finished 2024-09-16T08:50:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[40028]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi160-75741-40"}]': finished 2024-09-16T08:50:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[40028]: osdmap e242: 8 total, 8 up, 8 in 2024-09-16T08:50:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[40028]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi160-75741-41"} : dispatch 2024-09-16T08:50:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi160-75741-41"} : dispatch 2024-09-16T08:50:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi160-75741-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:25.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:26.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:26 smithi169 ceph-mon[33465]: pgmap v312: 228 pgs: 228 active+clean; 4.6 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:26 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:26 smithi169 ceph-mon[33465]: from='client.57125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi160-75757-49"}]': finished 2024-09-16T08:50:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi160-75741-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:26 smithi169 ceph-mon[33465]: osdmap e243: 8 total, 8 up, 8 in 2024-09-16T08:50:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-33"} : dispatch 2024-09-16T08:50:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi160-75741-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi160-75741-41"} : dispatch 2024-09-16T08:50:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:26 smithi169 ceph-mon[33465]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:26.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[30137]: pgmap v312: 228 pgs: 228 active+clean; 4.6 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[30137]: from='client.57125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi160-75757-49"}]': finished 2024-09-16T08:50:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi160-75741-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[30137]: osdmap e243: 8 total, 8 up, 8 in 2024-09-16T08:50:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-33"} : dispatch 2024-09-16T08:50:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi160-75741-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi160-75741-41"} : dispatch 2024-09-16T08:50:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[30137]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:26.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:26.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[40028]: pgmap v312: 228 pgs: 228 active+clean; 4.6 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:26.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:26.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[40028]: from='client.57125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi160-75757-49"}]': finished 2024-09-16T08:50:26.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi160-75741-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:26.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:26.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[40028]: osdmap e243: 8 total, 8 up, 8 in 2024-09-16T08:50:26.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-33"} : dispatch 2024-09-16T08:50:26.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/858138463' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:26.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi160-75741-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi160-75741-41"} : dispatch 2024-09-16T08:50:26.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:26 smithi160 ceph-mon[40028]: from='client.57125 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi160-75757-49"} : dispatch 2024-09-16T08:50:27.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:27 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-33"}]': finished 2024-09-16T08:50:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:27 smithi169 ceph-mon[33465]: from='client.57125 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi160-75757-49"}]': finished 2024-09-16T08:50:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:27 smithi169 ceph-mon[33465]: osdmap e244: 8 total, 8 up, 8 in 2024-09-16T08:50:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:27 smithi169 ceph-mon[33465]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:27 smithi169 ceph-mon[33465]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi160-75757-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:27 smithi169 ceph-mon[33465]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi160-75757-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:27.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-33"}]': finished 2024-09-16T08:50:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[30137]: from='client.57125 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi160-75757-49"}]': finished 2024-09-16T08:50:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[30137]: osdmap e244: 8 total, 8 up, 8 in 2024-09-16T08:50:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[30137]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[30137]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi160-75757-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[30137]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi160-75757-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:27.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:27.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-33"}]': finished 2024-09-16T08:50:27.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[40028]: from='client.57125 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi160-75757-49"}]': finished 2024-09-16T08:50:27.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[40028]: osdmap e244: 8 total, 8 up, 8 in 2024-09-16T08:50:27.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:27.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[40028]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:27.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:27.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[40028]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:27.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi160-75757-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:27.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:27 smithi160 ceph-mon[40028]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi160-75757-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:28.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:28 smithi169 ceph-mon[33465]: pgmap v315: 228 pgs: 228 active+clean; 4.6 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:28.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi160-75741-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi160-75741-41"}]': finished 2024-09-16T08:50:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:28 smithi169 ceph-mon[33465]: from='client.57137 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi160-75757-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:28 smithi169 ceph-mon[33465]: osdmap e245: 8 total, 8 up, 8 in 2024-09-16T08:50:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi160-75757-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:28 smithi169 ceph-mon[33465]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi160-75757-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:28.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:28 smithi160 ceph-mon[30137]: pgmap v315: 228 pgs: 228 active+clean; 4.6 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:28.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi160-75741-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi160-75741-41"}]': finished 2024-09-16T08:50:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:28 smithi160 ceph-mon[30137]: from='client.57137 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi160-75757-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:28 smithi160 ceph-mon[30137]: osdmap e245: 8 total, 8 up, 8 in 2024-09-16T08:50:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi160-75757-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:28 smithi160 ceph-mon[30137]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi160-75757-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:28 smithi160 ceph-mon[40028]: pgmap v315: 228 pgs: 228 active+clean; 4.6 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi160-75741-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi160-75741-41"}]': finished 2024-09-16T08:50:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:28 smithi160 ceph-mon[40028]: from='client.57137 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi160-75757-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:28 smithi160 ceph-mon[40028]: osdmap e245: 8 total, 8 up, 8 in 2024-09-16T08:50:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi160-75757-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:28 smithi160 ceph-mon[40028]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi160-75757-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:29 smithi160 ceph-mon[30137]: osdmap e246: 8 total, 8 up, 8 in 2024-09-16T08:50:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:29 smithi160 ceph-mon[30137]: from='client.57137 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi160-75757-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi160-75757-50"}]': finished 2024-09-16T08:50:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:29 smithi160 ceph-mon[30137]: osdmap e247: 8 total, 8 up, 8 in 2024-09-16T08:50:29.600 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi160-75741-41"} : dispatch 2024-09-16T08:50:29.600 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:29 smithi160 ceph-mon[40028]: osdmap e246: 8 total, 8 up, 8 in 2024-09-16T08:50:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:29 smithi160 ceph-mon[40028]: from='client.57137 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi160-75757-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi160-75757-50"}]': finished 2024-09-16T08:50:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:29.601 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:29 smithi160 ceph-mon[40028]: osdmap e247: 8 total, 8 up, 8 in 2024-09-16T08:50:29.601 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi160-75741-41"} : dispatch 2024-09-16T08:50:29.601 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:29 smithi169 ceph-mon[33465]: osdmap e246: 8 total, 8 up, 8 in 2024-09-16T08:50:29.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:29.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:29 smithi169 ceph-mon[33465]: from='client.57137 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi160-75757-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi160-75757-50"}]': finished 2024-09-16T08:50:29.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:29.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:29 smithi169 ceph-mon[33465]: osdmap e247: 8 total, 8 up, 8 in 2024-09-16T08:50:29.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi160-75741-41"} : dispatch 2024-09-16T08:50:29.602 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:30.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:30 smithi169 ceph-mon[33465]: pgmap v318: 236 pgs: 32 creating+peering, 204 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T08:50:30.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi160-75741-41"}]': finished 2024-09-16T08:50:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:30 smithi169 ceph-mon[33465]: osdmap e248: 8 total, 8 up, 8 in 2024-09-16T08:50:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi160-75741-41"} : dispatch 2024-09-16T08:50:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-35"} : dispatch 2024-09-16T08:50:30.621 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:30 smithi160 ceph-mon[30137]: pgmap v318: 236 pgs: 32 creating+peering, 204 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T08:50:30.621 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi160-75741-41"}]': finished 2024-09-16T08:50:30.622 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:30.622 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:30 smithi160 ceph-mon[30137]: osdmap e248: 8 total, 8 up, 8 in 2024-09-16T08:50:30.622 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi160-75741-41"} : dispatch 2024-09-16T08:50:30.622 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-35"} : dispatch 2024-09-16T08:50:30.622 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:30 smithi160 ceph-mon[40028]: pgmap v318: 236 pgs: 32 creating+peering, 204 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T08:50:30.622 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi160-75741-41"}]': finished 2024-09-16T08:50:30.622 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:30.622 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:30 smithi160 ceph-mon[40028]: osdmap e248: 8 total, 8 up, 8 in 2024-09-16T08:50:30.622 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi160-75741-41"} : dispatch 2024-09-16T08:50:30.622 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-35"} : dispatch 2024-09-16T08:50:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:32 smithi169 ceph-mon[33465]: pgmap v321: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:50:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi160-75741-41"}]': finished 2024-09-16T08:50:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-35"}]': finished 2024-09-16T08:50:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:32 smithi169 ceph-mon[33465]: osdmap e249: 8 total, 8 up, 8 in 2024-09-16T08:50:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-35", "mode": "writeback"} : dispatch 2024-09-16T08:50:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:32 smithi169 ceph-mon[33465]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:32 smithi169 ceph-mon[33465]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:32.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:32.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:32 smithi169 ceph-mon[33465]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:32.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi160-75741-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:32.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:32 smithi169 ceph-mon[33465]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi160-75741-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:32.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:32 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[30137]: pgmap v321: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:50:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi160-75741-41"}]': finished 2024-09-16T08:50:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-35"}]': finished 2024-09-16T08:50:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[30137]: osdmap e249: 8 total, 8 up, 8 in 2024-09-16T08:50:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-35", "mode": "writeback"} : dispatch 2024-09-16T08:50:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[30137]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[30137]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:32.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[30137]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:32.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi160-75741-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:32.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[30137]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi160-75741-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:32.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[40028]: pgmap v321: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:50:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3657549558' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi160-75741-41"}]': finished 2024-09-16T08:50:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-35"}]': finished 2024-09-16T08:50:32.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[40028]: osdmap e249: 8 total, 8 up, 8 in 2024-09-16T08:50:32.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-35", "mode": "writeback"} : dispatch 2024-09-16T08:50:32.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:32.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[40028]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:32.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:32.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[40028]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:32.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:32.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[40028]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:32.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi160-75741-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:32.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[40028]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi160-75741-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:32.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:32 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:33.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:33 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:33.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-35", "mode": "writeback"}]': finished 2024-09-16T08:50:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:33 smithi169 ceph-mon[33465]: from='client.57137 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi160-75757-50"}]': finished 2024-09-16T08:50:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:33 smithi169 ceph-mon[33465]: from='client.57143 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi160-75741-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:33 smithi169 ceph-mon[33465]: osdmap e250: 8 total, 8 up, 8 in 2024-09-16T08:50:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi160-75741-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:33 smithi169 ceph-mon[33465]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi160-75741-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:33 smithi169 ceph-mon[33465]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:33.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-35", "mode": "writeback"}]': finished 2024-09-16T08:50:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[30137]: from='client.57137 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi160-75757-50"}]': finished 2024-09-16T08:50:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[30137]: from='client.57143 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi160-75741-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[30137]: osdmap e250: 8 total, 8 up, 8 in 2024-09-16T08:50:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi160-75741-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[30137]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi160-75741-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[30137]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-35", "mode": "writeback"}]': finished 2024-09-16T08:50:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[40028]: from='client.57137 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi160-75757-50"}]': finished 2024-09-16T08:50:33.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[40028]: from='client.57143 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi160-75741-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:33.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[40028]: osdmap e250: 8 total, 8 up, 8 in 2024-09-16T08:50:33.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi160-75741-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:33.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/647601509' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:33.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[40028]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi160-75741-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:33.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:33 smithi160 ceph-mon[40028]: from='client.57137 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi160-75757-50"} : dispatch 2024-09-16T08:50:34.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:34 smithi169 ceph-mon[33465]: pgmap v324: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:34.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:34 smithi169 ceph-mon[33465]: from='client.57137 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi160-75757-50"}]': finished 2024-09-16T08:50:34.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:34 smithi169 ceph-mon[33465]: osdmap e251: 8 total, 8 up, 8 in 2024-09-16T08:50:34.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:50:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:50:34.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:34 smithi160 ceph-mon[30137]: pgmap v324: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:34.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:34 smithi160 ceph-mon[30137]: from='client.57137 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi160-75757-50"}]': finished 2024-09-16T08:50:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:34 smithi160 ceph-mon[30137]: osdmap e251: 8 total, 8 up, 8 in 2024-09-16T08:50:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:50:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:50:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:34 smithi160 ceph-mon[40028]: pgmap v324: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:34 smithi160 ceph-mon[40028]: from='client.57137 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi160-75757-50"}]': finished 2024-09-16T08:50:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:34 smithi160 ceph-mon[40028]: osdmap e251: 8 total, 8 up, 8 in 2024-09-16T08:50:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:50:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:50:35.245 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:50:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:50:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:50:35.245 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:50:34 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:50:35.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:35 smithi169 ceph-mon[33465]: from='client.57143 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi160-75741-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi160-75741-42"}]': finished 2024-09-16T08:50:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:35 smithi169 ceph-mon[33465]: osdmap e252: 8 total, 8 up, 8 in 2024-09-16T08:50:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2291219955' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:35 smithi169 ceph-mon[33465]: from='client.48316 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:35 smithi160 ceph-mon[30137]: from='client.57143 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi160-75741-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi160-75741-42"}]': finished 2024-09-16T08:50:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:35 smithi160 ceph-mon[30137]: osdmap e252: 8 total, 8 up, 8 in 2024-09-16T08:50:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2291219955' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:35.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:35 smithi160 ceph-mon[30137]: from='client.48316 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:35.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:35.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:35 smithi160 ceph-mon[40028]: from='client.57143 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi160-75741-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi160-75741-42"}]': finished 2024-09-16T08:50:35.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:35 smithi160 ceph-mon[40028]: osdmap e252: 8 total, 8 up, 8 in 2024-09-16T08:50:35.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2291219955' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:35.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:35 smithi160 ceph-mon[40028]: from='client.48316 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:35.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:36.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:36 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:36.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:36 smithi169 ceph-mon[33465]: pgmap v327: 268 pgs: 40 unknown, 228 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:50:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:36 smithi169 ceph-mon[33465]: from='client.48316 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:36 smithi169 ceph-mon[33465]: osdmap e253: 8 total, 8 up, 8 in 2024-09-16T08:50:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-35"} : dispatch 2024-09-16T08:50:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:36 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:36 smithi160 ceph-mon[30137]: pgmap v327: 268 pgs: 40 unknown, 228 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:50:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:36 smithi160 ceph-mon[30137]: from='client.48316 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:36 smithi160 ceph-mon[30137]: osdmap e253: 8 total, 8 up, 8 in 2024-09-16T08:50:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-35"} : dispatch 2024-09-16T08:50:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:36 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:36 smithi160 ceph-mon[40028]: pgmap v327: 268 pgs: 40 unknown, 228 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:50:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:36 smithi160 ceph-mon[40028]: from='client.48316 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi160-75757-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:36 smithi160 ceph-mon[40028]: osdmap e253: 8 total, 8 up, 8 in 2024-09-16T08:50:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-35"} : dispatch 2024-09-16T08:50:37.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-35"}]': finished 2024-09-16T08:50:37.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:37 smithi169 ceph-mon[33465]: osdmap e254: 8 total, 8 up, 8 in 2024-09-16T08:50:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:37 smithi169 ceph-mon[33465]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi160-75757-52"} : dispatch 2024-09-16T08:50:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi160-75757-52"} : dispatch 2024-09-16T08:50:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi160-75757-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:37.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-35"}]': finished 2024-09-16T08:50:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:37 smithi160 ceph-mon[30137]: osdmap e254: 8 total, 8 up, 8 in 2024-09-16T08:50:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:37 smithi160 ceph-mon[30137]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi160-75757-52"} : dispatch 2024-09-16T08:50:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi160-75757-52"} : dispatch 2024-09-16T08:50:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi160-75757-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-35"}]': finished 2024-09-16T08:50:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:37 smithi160 ceph-mon[40028]: osdmap e254: 8 total, 8 up, 8 in 2024-09-16T08:50:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:37 smithi160 ceph-mon[40028]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi160-75757-52"} : dispatch 2024-09-16T08:50:37.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi160-75757-52"} : dispatch 2024-09-16T08:50:37.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi160-75757-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:38.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7012 ms) 2024-09-16T08:50:38.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-09-16T08:50:38.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6962 ms) 2024-09-16T08:50:38.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-09-16T08:50:38.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7011 ms) 2024-09-16T08:50:38.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-09-16T08:50:38.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7006 ms) 2024-09-16T08:50:38.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-09-16T08:50:38.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7014 ms) 2024-09-16T08:50:38.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-09-16T08:50:38.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7025 ms) 2024-09-16T08:50:38.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-09-16T08:50:38.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7049 ms) 2024-09-16T08:50:38.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-09-16T08:50:38.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7016 ms) 2024-09-16T08:50:38.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-09-16T08:50:38.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6903 ms) 2024-09-16T08:50:38.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-09-16T08:50:38.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7043 ms) 2024-09-16T08:50:38.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-09-16T08:50:38.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7024 ms) 2024-09-16T08:50:38.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-09-16T08:50:38.278 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6944 ms) 2024-09-16T08:50:38.278 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-09-16T08:50:38.278 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6982 ms) 2024-09-16T08:50:38.278 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-09-16T08:50:38.278 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7040 ms) 2024-09-16T08:50:38.278 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-09-16T08:50:38.278 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7031 ms) 2024-09-16T08:50:38.278 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-09-16T08:50:38.278 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7044 ms) 2024-09-16T08:50:38.278 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112108 ms total) 2024-09-16T08:50:38.278 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: 2024-09-16T08:50:38.278 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [----------] Global test environment tear-down 2024-09-16T08:50:38.278 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (189761 ms total) 2024-09-16T08:50:38.279 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ PASSED ] 42 tests. 2024-09-16T08:50:38.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:38 smithi169 ceph-mon[33465]: pgmap v330: 228 pgs: 228 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:50:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:38 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:38 smithi169 ceph-mon[33465]: from='client.57143 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi160-75741-42"}]': finished 2024-09-16T08:50:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi160-75757-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:38 smithi169 ceph-mon[33465]: osdmap e255: 8 total, 8 up, 8 in 2024-09-16T08:50:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi160-75757-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi160-75757-52"} : dispatch 2024-09-16T08:50:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:38 smithi169 ceph-mon[33465]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:38.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[30137]: pgmap v330: 228 pgs: 228 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:50:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[30137]: from='client.57143 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi160-75741-42"}]': finished 2024-09-16T08:50:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi160-75757-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[30137]: osdmap e255: 8 total, 8 up, 8 in 2024-09-16T08:50:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi160-75757-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi160-75757-52"} : dispatch 2024-09-16T08:50:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[30137]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[40028]: pgmap v330: 228 pgs: 228 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:50:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[40028]: from='client.57143 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi160-75741-42"}]': finished 2024-09-16T08:50:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi160-75757-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[40028]: osdmap e255: 8 total, 8 up, 8 in 2024-09-16T08:50:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi160-75757-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi160-75757-52"} : dispatch 2024-09-16T08:50:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1911171765' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[40028]: from='client.57143 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi160-75741-42"} : dispatch 2024-09-16T08:50:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:39.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:39 smithi169 ceph-mon[33465]: from='client.57143 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi160-75741-42"}]': finished 2024-09-16T08:50:39.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:39.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:39 smithi169 ceph-mon[33465]: osdmap e256: 8 total, 8 up, 8 in 2024-09-16T08:50:39.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-35"} : dispatch 2024-09-16T08:50:39.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:39 smithi160 ceph-mon[30137]: from='client.57143 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi160-75741-42"}]': finished 2024-09-16T08:50:39.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:39.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:39 smithi160 ceph-mon[30137]: osdmap e256: 8 total, 8 up, 8 in 2024-09-16T08:50:39.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-35"} : dispatch 2024-09-16T08:50:39.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:39 smithi160 ceph-mon[40028]: from='client.57143 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi160-75741-42"}]': finished 2024-09-16T08:50:39.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:39.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:39 smithi160 ceph-mon[40028]: osdmap e256: 8 total, 8 up, 8 in 2024-09-16T08:50:39.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-35"} : dispatch 2024-09-16T08:50:40.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:40 smithi169 ceph-mon[33465]: pgmap v333: 228 pgs: 228 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-16T08:50:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:40 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi160-75757-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi160-75757-52"}]': finished 2024-09-16T08:50:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-35"}]': finished 2024-09-16T08:50:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:40 smithi169 ceph-mon[33465]: osdmap e257: 8 total, 8 up, 8 in 2024-09-16T08:50:40.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:40 smithi160 ceph-mon[30137]: pgmap v333: 228 pgs: 228 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-16T08:50:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:40 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi160-75757-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi160-75757-52"}]': finished 2024-09-16T08:50:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-35"}]': finished 2024-09-16T08:50:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:40 smithi160 ceph-mon[30137]: osdmap e257: 8 total, 8 up, 8 in 2024-09-16T08:50:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:40 smithi160 ceph-mon[40028]: pgmap v333: 228 pgs: 228 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-16T08:50:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:40 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi160-75757-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi160-75757-52"}]': finished 2024-09-16T08:50:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-35"}]': finished 2024-09-16T08:50:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:40 smithi160 ceph-mon[40028]: osdmap e257: 8 total, 8 up, 8 in 2024-09-16T08:50:41.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:41 smithi169 ceph-mon[33465]: osdmap e258: 8 total, 8 up, 8 in 2024-09-16T08:50:41.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:41 smithi169 ceph-mon[33465]: osdmap e259: 8 total, 8 up, 8 in 2024-09-16T08:50:41.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi160-75757-52"} : dispatch 2024-09-16T08:50:41.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:41 smithi160 ceph-mon[30137]: osdmap e258: 8 total, 8 up, 8 in 2024-09-16T08:50:41.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:41 smithi160 ceph-mon[30137]: osdmap e259: 8 total, 8 up, 8 in 2024-09-16T08:50:41.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:41.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi160-75757-52"} : dispatch 2024-09-16T08:50:41.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:41 smithi160 ceph-mon[40028]: osdmap e258: 8 total, 8 up, 8 in 2024-09-16T08:50:41.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:41 smithi160 ceph-mon[40028]: osdmap e259: 8 total, 8 up, 8 in 2024-09-16T08:50:41.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:41.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi160-75757-52"} : dispatch 2024-09-16T08:50:42.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:42 smithi169 ceph-mon[33465]: pgmap v336: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-16T08:50:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi160-75757-52"}]': finished 2024-09-16T08:50:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:42 smithi169 ceph-mon[33465]: osdmap e260: 8 total, 8 up, 8 in 2024-09-16T08:50:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi160-75757-52"} : dispatch 2024-09-16T08:50:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:42.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:42 smithi160 ceph-mon[30137]: pgmap v336: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-16T08:50:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi160-75757-52"}]': finished 2024-09-16T08:50:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:42 smithi160 ceph-mon[30137]: osdmap e260: 8 total, 8 up, 8 in 2024-09-16T08:50:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi160-75757-52"} : dispatch 2024-09-16T08:50:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:42 smithi160 ceph-mon[40028]: pgmap v336: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-16T08:50:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi160-75757-52"}]': finished 2024-09-16T08:50:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:42 smithi160 ceph-mon[40028]: osdmap e260: 8 total, 8 up, 8 in 2024-09-16T08:50:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi160-75757-52"} : dispatch 2024-09-16T08:50:42.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: pgmap v339: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi160-75757-52"}]': finished 2024-09-16T08:50:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: osdmap e261: 8 total, 8 up, 8 in 2024-09-16T08:50:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-37"} : dispatch 2024-09-16T08:50:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-37"}]': finished 2024-09-16T08:50:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: from='client.57161 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: osdmap e262: 8 total, 8 up, 8 in 2024-09-16T08:50:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-37", "mode": "writeback"} : dispatch 2024-09-16T08:50:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi160-75757-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-mon[33465]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi160-75757-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: pgmap v339: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi160-75757-52"}]': finished 2024-09-16T08:50:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: osdmap e261: 8 total, 8 up, 8 in 2024-09-16T08:50:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-37"} : dispatch 2024-09-16T08:50:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-37"}]': finished 2024-09-16T08:50:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: from='client.57161 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: osdmap e262: 8 total, 8 up, 8 in 2024-09-16T08:50:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-37", "mode": "writeback"} : dispatch 2024-09-16T08:50:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi160-75757-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[30137]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi160-75757-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: pgmap v339: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1216776277' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi160-75757-52"}]': finished 2024-09-16T08:50:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: osdmap e261: 8 total, 8 up, 8 in 2024-09-16T08:50:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-37"} : dispatch 2024-09-16T08:50:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:44.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-37"}]': finished 2024-09-16T08:50:44.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: from='client.57161 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:44.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: osdmap e262: 8 total, 8 up, 8 in 2024-09-16T08:50:44.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-37", "mode": "writeback"} : dispatch 2024-09-16T08:50:44.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi160-75757-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:44.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-mon[40028]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi160-75757-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:45.286 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:50:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:50:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:50:45.289 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:50:44 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:50:45.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:45 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:45.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-37", "mode": "writeback"}]': finished 2024-09-16T08:50:45.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:45 smithi169 ceph-mon[33465]: osdmap e263: 8 total, 8 up, 8 in 2024-09-16T08:50:45.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:45.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:45 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:45.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-37", "mode": "writeback"}]': finished 2024-09-16T08:50:45.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:45 smithi160 ceph-mon[40028]: osdmap e263: 8 total, 8 up, 8 in 2024-09-16T08:50:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:45 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-37", "mode": "writeback"}]': finished 2024-09-16T08:50:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:45 smithi160 ceph-mon[30137]: osdmap e263: 8 total, 8 up, 8 in 2024-09-16T08:50:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:46.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:46 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:46.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:46 smithi169 ceph-mon[33465]: pgmap v342: 228 pgs: 228 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:46 smithi169 ceph-mon[33465]: from='client.57161 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi160-75757-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi160-75757-53"}]': finished 2024-09-16T08:50:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:46 smithi169 ceph-mon[33465]: osdmap e264: 8 total, 8 up, 8 in 2024-09-16T08:50:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-37"} : dispatch 2024-09-16T08:50:46.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:46 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:46 smithi160 ceph-mon[30137]: pgmap v342: 228 pgs: 228 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:46 smithi160 ceph-mon[30137]: from='client.57161 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi160-75757-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi160-75757-53"}]': finished 2024-09-16T08:50:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:46 smithi160 ceph-mon[30137]: osdmap e264: 8 total, 8 up, 8 in 2024-09-16T08:50:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-37"} : dispatch 2024-09-16T08:50:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:46 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:46 smithi160 ceph-mon[40028]: pgmap v342: 228 pgs: 228 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:46 smithi160 ceph-mon[40028]: from='client.57161 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi160-75757-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi160-75757-53"}]': finished 2024-09-16T08:50:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:46 smithi160 ceph-mon[40028]: osdmap e264: 8 total, 8 up, 8 in 2024-09-16T08:50:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-37"} : dispatch 2024-09-16T08:50:47.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:47 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-37"}]': finished 2024-09-16T08:50:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:47 smithi169 ceph-mon[33465]: osdmap e265: 8 total, 8 up, 8 in 2024-09-16T08:50:47.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:47 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:47.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-37"}]': finished 2024-09-16T08:50:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:47 smithi160 ceph-mon[40028]: osdmap e265: 8 total, 8 up, 8 in 2024-09-16T08:50:47.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:47 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:47.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-37"}]': finished 2024-09-16T08:50:47.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:47 smithi160 ceph-mon[30137]: osdmap e265: 8 total, 8 up, 8 in 2024-09-16T08:50:48.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:48 smithi169 ceph-mon[33465]: pgmap v345: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:48 smithi169 ceph-mon[33465]: osdmap e266: 8 total, 8 up, 8 in 2024-09-16T08:50:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:48 smithi169 ceph-mon[33465]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:48.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:48 smithi160 ceph-mon[40028]: pgmap v345: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:48 smithi160 ceph-mon[40028]: osdmap e266: 8 total, 8 up, 8 in 2024-09-16T08:50:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:48 smithi160 ceph-mon[40028]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:48 smithi160 ceph-mon[30137]: pgmap v345: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:50:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:48 smithi160 ceph-mon[30137]: osdmap e266: 8 total, 8 up, 8 in 2024-09-16T08:50:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:48 smithi160 ceph-mon[30137]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:50:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:50:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[40028]: pgmap v348: 196 pgs: 196 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:50:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[40028]: from='client.57161 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53"}]': finished 2024-09-16T08:50:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[40028]: osdmap e267: 8 total, 8 up, 8 in 2024-09-16T08:50:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[40028]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:49.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:50:49.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:50:49.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[30137]: pgmap v348: 196 pgs: 196 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:50:49.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:49.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[30137]: from='client.57161 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53"}]': finished 2024-09-16T08:50:49.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[30137]: osdmap e267: 8 total, 8 up, 8 in 2024-09-16T08:50:49.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:49.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:49.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:49 smithi160 ceph-mon[30137]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:50.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:50:50.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:50:50.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:49 smithi169 ceph-mon[33465]: pgmap v348: 196 pgs: 196 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:50:50.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:49 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:50.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:49 smithi169 ceph-mon[33465]: from='client.57161 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi160-75757-53"}]': finished 2024-09-16T08:50:50.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:49 smithi169 ceph-mon[33465]: osdmap e267: 8 total, 8 up, 8 in 2024-09-16T08:50:50.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:50.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2813590881' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:50.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:49 smithi169 ceph-mon[33465]: from='client.57161 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi160-75757-53"} : dispatch 2024-09-16T08:50:51.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:51.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[40028]: from='client.57161 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi160-75757-53"}]': finished 2024-09-16T08:50:51.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[40028]: osdmap e268: 8 total, 8 up, 8 in 2024-09-16T08:50:51.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:51.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:51.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[40028]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:51.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:51.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[40028]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:51.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi160-75757-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:51.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[40028]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi160-75757-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:51.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:51.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[30137]: from='client.57161 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi160-75757-53"}]': finished 2024-09-16T08:50:51.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[30137]: osdmap e268: 8 total, 8 up, 8 in 2024-09-16T08:50:51.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:51.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:51.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[30137]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:51.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:51.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[30137]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:51.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi160-75757-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:51.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:51 smithi160 ceph-mon[30137]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi160-75757-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:51.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:51.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:51 smithi169 ceph-mon[33465]: from='client.57161 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi160-75757-53"}]': finished 2024-09-16T08:50:51.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:51 smithi169 ceph-mon[33465]: osdmap e268: 8 total, 8 up, 8 in 2024-09-16T08:50:51.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:51.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:51.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:51 smithi169 ceph-mon[33465]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:51.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:51.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:51 smithi169 ceph-mon[33465]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:51.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi160-75757-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:51.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:51 smithi169 ceph-mon[33465]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi160-75757-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:52.447 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:52 smithi160 ceph-mon[40028]: pgmap v351: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:50:52.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:52.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:52 smithi160 ceph-mon[40028]: from='client.57167 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi160-75757-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:52.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:52 smithi160 ceph-mon[40028]: osdmap e269: 8 total, 8 up, 8 in 2024-09-16T08:50:52.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-39"} : dispatch 2024-09-16T08:50:52.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi160-75757-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:52.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:52 smithi160 ceph-mon[40028]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi160-75757-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:52.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:52 smithi160 ceph-mon[30137]: pgmap v351: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:50:52.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:52.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:52 smithi160 ceph-mon[30137]: from='client.57167 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi160-75757-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:52.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:52 smithi160 ceph-mon[30137]: osdmap e269: 8 total, 8 up, 8 in 2024-09-16T08:50:52.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-39"} : dispatch 2024-09-16T08:50:52.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi160-75757-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:52.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:52 smithi160 ceph-mon[30137]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi160-75757-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:52 smithi169 ceph-mon[33465]: pgmap v351: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:50:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:52 smithi169 ceph-mon[33465]: from='client.57167 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi160-75757-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:52 smithi169 ceph-mon[33465]: osdmap e269: 8 total, 8 up, 8 in 2024-09-16T08:50:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-39"} : dispatch 2024-09-16T08:50:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi160-75757-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:52 smithi169 ceph-mon[33465]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi160-75757-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:53.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-39"}]': finished 2024-09-16T08:50:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:53 smithi160 ceph-mon[30137]: osdmap e270: 8 total, 8 up, 8 in 2024-09-16T08:50:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-39", "mode": "writeback"} : dispatch 2024-09-16T08:50:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:53 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:53 smithi160 ceph-mon[30137]: from='client.57167 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi160-75757-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi160-75757-54"}]': finished 2024-09-16T08:50:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-39", "mode": "writeback"}]': finished 2024-09-16T08:50:53.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:53 smithi160 ceph-mon[30137]: osdmap e271: 8 total, 8 up, 8 in 2024-09-16T08:50:53.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-39"}]': finished 2024-09-16T08:50:53.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:53 smithi160 ceph-mon[40028]: osdmap e270: 8 total, 8 up, 8 in 2024-09-16T08:50:53.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-39", "mode": "writeback"} : dispatch 2024-09-16T08:50:53.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:53 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:53.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:53 smithi160 ceph-mon[40028]: from='client.57167 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi160-75757-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi160-75757-54"}]': finished 2024-09-16T08:50:53.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-39", "mode": "writeback"}]': finished 2024-09-16T08:50:53.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:53 smithi160 ceph-mon[40028]: osdmap e271: 8 total, 8 up, 8 in 2024-09-16T08:50:53.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-39"}]': finished 2024-09-16T08:50:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:53 smithi169 ceph-mon[33465]: osdmap e270: 8 total, 8 up, 8 in 2024-09-16T08:50:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-39", "mode": "writeback"} : dispatch 2024-09-16T08:50:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:53 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:50:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:53 smithi169 ceph-mon[33465]: from='client.57167 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi160-75757-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi160-75757-54"}]': finished 2024-09-16T08:50:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-39", "mode": "writeback"}]': finished 2024-09-16T08:50:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:53 smithi169 ceph-mon[33465]: osdmap e271: 8 total, 8 up, 8 in 2024-09-16T08:50:54.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:54 smithi160 ceph-mon[30137]: pgmap v354: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:54.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:54.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:54 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:54.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:54.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:54 smithi160 ceph-mon[30137]: osdmap e272: 8 total, 8 up, 8 in 2024-09-16T08:50:54.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-39"} : dispatch 2024-09-16T08:50:54.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:54 smithi160 ceph-mon[40028]: pgmap v354: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:54.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:54.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:54 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:54.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:54.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:54 smithi160 ceph-mon[40028]: osdmap e272: 8 total, 8 up, 8 in 2024-09-16T08:50:54.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-39"} : dispatch 2024-09-16T08:50:54.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:54 smithi169 ceph-mon[33465]: pgmap v354: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:50:54.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:50:54.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:54 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:50:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:50:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:54 smithi169 ceph-mon[33465]: osdmap e272: 8 total, 8 up, 8 in 2024-09-16T08:50:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-39"} : dispatch 2024-09-16T08:50:55.349 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:50:54 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:50:55.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:55 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:55.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-39"}]': finished 2024-09-16T08:50:55.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:55 smithi169 ceph-mon[33465]: osdmap e273: 8 total, 8 up, 8 in 2024-09-16T08:50:55.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:55.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:55 smithi169 ceph-mon[33465]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:55.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:55 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:55.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-39"}]': finished 2024-09-16T08:50:55.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:55 smithi160 ceph-mon[30137]: osdmap e273: 8 total, 8 up, 8 in 2024-09-16T08:50:55.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:55.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:55 smithi160 ceph-mon[30137]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:55.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:55 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:50:55.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-39"}]': finished 2024-09-16T08:50:55.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:55 smithi160 ceph-mon[40028]: osdmap e273: 8 total, 8 up, 8 in 2024-09-16T08:50:55.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:55.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:55 smithi160 ceph-mon[40028]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:55.449 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:50:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:50:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:50:56.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-09-16T08:50:56.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-09-16T08:50:56.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-09-16T08:50:56.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: seed 76079 2024-09-16T08:50:56.184 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-09-16T08:50:56.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (375 ms) 2024-09-16T08:50:56.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-09-16T08:50:56.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (10595 ms) 2024-09-16T08:50:56.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-09-16T08:50:56.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (37 ms) 2024-09-16T08:50:56.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (11007 ms total) 2024-09-16T08:50:56.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: 2024-09-16T08:50:56.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-09-16T08:50:56.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-09-16T08:50:56.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (7041 ms) 2024-09-16T08:50:56.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-09-16T08:50:56.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (7966 ms) 2024-09-16T08:50:56.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-09-16T08:50:56.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (9975 ms) 2024-09-16T08:50:56.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-09-16T08:50:56.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: my_snaps [3] 2024-09-16T08:50:56.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: my_snaps [4,3] 2024-09-16T08:50:56.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: my_snaps [5,4,3] 2024-09-16T08:50:56.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-09-16T08:50:56.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: promoting some heads 2024-09-16T08:50:56.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: promoting from clones for snap 6 2024-09-16T08:50:56.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: promoting from clones for snap 5 2024-09-16T08:50:56.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: promoting from clones for snap 4 2024-09-16T08:50:56.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: promoting from clones for snap 3 2024-09-16T08:50:56.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: waiting for scrubs... 2024-09-16T08:50:56.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: done waiting 2024-09-16T08:50:56.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46072 ms) 2024-09-16T08:50:56.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-09-16T08:50:56.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9958 ms) 2024-09-16T08:50:56.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-09-16T08:50:56.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7996 ms) 2024-09-16T08:50:56.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-09-16T08:50:56.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8035 ms) 2024-09-16T08:50:56.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-09-16T08:50:56.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8057 ms) 2024-09-16T08:50:56.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-09-16T08:50:56.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10023 ms) 2024-09-16T08:50:56.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-09-16T08:50:56.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8899 ms) 2024-09-16T08:50:56.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-09-16T08:50:56.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10064 ms) 2024-09-16T08:50:56.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-09-16T08:50:56.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (12959 ms) 2024-09-16T08:50:56.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-09-16T08:50:56.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7993 ms) 2024-09-16T08:50:56.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-09-16T08:50:56.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8047 ms) 2024-09-16T08:50:56.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-09-16T08:50:56.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13067 ms) 2024-09-16T08:50:56.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-09-16T08:50:56.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7885 ms) 2024-09-16T08:50:56.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-09-16T08:50:56.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8015 ms) 2024-09-16T08:50:56.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:56 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:56.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:56 smithi169 ceph-mon[33465]: pgmap v357: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 81 op/s 2024-09-16T08:50:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:56 smithi169 ceph-mon[33465]: from='client.57167 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi160-75757-54"}]': finished 2024-09-16T08:50:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:56 smithi169 ceph-mon[33465]: osdmap e274: 8 total, 8 up, 8 in 2024-09-16T08:50:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:56 smithi169 ceph-mon[33465]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:56.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:56 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:56 smithi160 ceph-mon[30137]: pgmap v357: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 81 op/s 2024-09-16T08:50:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:56 smithi160 ceph-mon[30137]: from='client.57167 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi160-75757-54"}]': finished 2024-09-16T08:50:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:56 smithi160 ceph-mon[30137]: osdmap e274: 8 total, 8 up, 8 in 2024-09-16T08:50:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:56 smithi160 ceph-mon[30137]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:56 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:50:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:56 smithi160 ceph-mon[40028]: pgmap v357: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 81 op/s 2024-09-16T08:50:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:56 smithi160 ceph-mon[40028]: from='client.57167 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi160-75757-54"}]': finished 2024-09-16T08:50:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:56 smithi160 ceph-mon[40028]: osdmap e274: 8 total, 8 up, 8 in 2024-09-16T08:50:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3340075448' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:56 smithi160 ceph-mon[40028]: from='client.57167 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi160-75757-54"} : dispatch 2024-09-16T08:50:58.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:58 smithi160 ceph-mon[40028]: pgmap v360: 196 pgs: 196 active+clean; 8.4 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-16T08:50:58.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:58 smithi160 ceph-mon[40028]: from='client.57167 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi160-75757-54"}]': finished 2024-09-16T08:50:58.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:58 smithi160 ceph-mon[40028]: osdmap e275: 8 total, 8 up, 8 in 2024-09-16T08:50:58.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:58.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi160-75757-55"} : dispatch 2024-09-16T08:50:58.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi160-75757-55"} : dispatch 2024-09-16T08:50:58.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi160-75757-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:58.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:58 smithi160 ceph-mon[30137]: pgmap v360: 196 pgs: 196 active+clean; 8.4 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-16T08:50:58.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:58 smithi160 ceph-mon[30137]: from='client.57167 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi160-75757-54"}]': finished 2024-09-16T08:50:58.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:58 smithi160 ceph-mon[30137]: osdmap e275: 8 total, 8 up, 8 in 2024-09-16T08:50:58.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:58.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi160-75757-55"} : dispatch 2024-09-16T08:50:58.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi160-75757-55"} : dispatch 2024-09-16T08:50:58.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi160-75757-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:58.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:58 smithi169 ceph-mon[33465]: pgmap v360: 196 pgs: 196 active+clean; 8.4 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-16T08:50:58.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:58 smithi169 ceph-mon[33465]: from='client.57167 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi160-75757-54"}]': finished 2024-09-16T08:50:58.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:58 smithi169 ceph-mon[33465]: osdmap e275: 8 total, 8 up, 8 in 2024-09-16T08:50:58.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:50:58.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi160-75757-55"} : dispatch 2024-09-16T08:50:58.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi160-75757-55"} : dispatch 2024-09-16T08:50:58.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi160-75757-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:50:59.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:59.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi160-75757-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:59 smithi169 ceph-mon[33465]: osdmap e276: 8 total, 8 up, 8 in 2024-09-16T08:50:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafePP_smithi160-75757-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi160-75757-55"} : dispatch 2024-09-16T08:50:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:59.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:50:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:59.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi160-75757-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:59 smithi160 ceph-mon[30137]: osdmap e276: 8 total, 8 up, 8 in 2024-09-16T08:50:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafePP_smithi160-75757-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi160-75757-55"} : dispatch 2024-09-16T08:50:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:50:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:50:59.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:50:59.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi160-75757-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:50:59.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:59 smithi160 ceph-mon[40028]: osdmap e276: 8 total, 8 up, 8 in 2024-09-16T08:50:59.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafePP_smithi160-75757-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi160-75757-55"} : dispatch 2024-09-16T08:50:59.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:50:59.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:50:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:51:00.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:00 smithi169 ceph-mon[33465]: pgmap v363: 228 pgs: 228 active+clean; 8.4 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:00.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:00 smithi169 ceph-mon[33465]: osdmap e277: 8 total, 8 up, 8 in 2024-09-16T08:51:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:51:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi160-75757-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi160-75757-55"}]': finished 2024-09-16T08:51:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_count","val": "2"}]': finished 2024-09-16T08:51:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:00 smithi169 ceph-mon[33465]: osdmap e278: 8 total, 8 up, 8 in 2024-09-16T08:51:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:51:00.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:00 smithi160 ceph-mon[40028]: pgmap v363: 228 pgs: 228 active+clean; 8.4 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:00.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:00 smithi160 ceph-mon[40028]: osdmap e277: 8 total, 8 up, 8 in 2024-09-16T08:51:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:51:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi160-75757-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi160-75757-55"}]': finished 2024-09-16T08:51:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_count","val": "2"}]': finished 2024-09-16T08:51:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:00 smithi160 ceph-mon[40028]: osdmap e278: 8 total, 8 up, 8 in 2024-09-16T08:51:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:51:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:00 smithi160 ceph-mon[30137]: pgmap v363: 228 pgs: 228 active+clean; 8.4 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:00 smithi160 ceph-mon[30137]: osdmap e277: 8 total, 8 up, 8 in 2024-09-16T08:51:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:51:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi160-75757-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi160-75757-55"}]': finished 2024-09-16T08:51:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_count","val": "2"}]': finished 2024-09-16T08:51:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:00 smithi160 ceph-mon[30137]: osdmap e278: 8 total, 8 up, 8 in 2024-09-16T08:51:00.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:51:02.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:02 smithi160 ceph-mon[30137]: pgmap v366: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:02.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:02 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:51:02.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:02 smithi160 ceph-mon[30137]: osdmap e279: 8 total, 8 up, 8 in 2024-09-16T08:51:02.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:02 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-16T08:51:02.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:02 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:02.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:02 smithi160 ceph-mon[40028]: pgmap v366: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:02.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:02 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:51:02.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:02 smithi160 ceph-mon[40028]: osdmap e279: 8 total, 8 up, 8 in 2024-09-16T08:51:02.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:02 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-16T08:51:02.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:02 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:02.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:02 smithi169 ceph-mon[33465]: pgmap v366: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:02.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:02 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:51:02.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:02 smithi169 ceph-mon[33465]: osdmap e279: 8 total, 8 up, 8 in 2024-09-16T08:51:02.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:02 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-16T08:51:02.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:02 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:03.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-16T08:51:03.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:03 smithi169 ceph-mon[33465]: osdmap e280: 8 total, 8 up, 8 in 2024-09-16T08:51:03.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi160-75757-55"} : dispatch 2024-09-16T08:51:03.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi160-75757-55"}]': finished 2024-09-16T08:51:03.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-16T08:51:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:03 smithi160 ceph-mon[30137]: osdmap e280: 8 total, 8 up, 8 in 2024-09-16T08:51:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi160-75757-55"} : dispatch 2024-09-16T08:51:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi160-75757-55"}]': finished 2024-09-16T08:51:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-16T08:51:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:03 smithi160 ceph-mon[40028]: osdmap e280: 8 total, 8 up, 8 in 2024-09-16T08:51:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi160-75757-55"} : dispatch 2024-09-16T08:51:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi160-75757-55"}]': finished 2024-09-16T08:51:04.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:04 smithi169 ceph-mon[33465]: pgmap v369: 228 pgs: 228 active+clean; 8.4 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:04.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:04 smithi169 ceph-mon[33465]: osdmap e281: 8 total, 8 up, 8 in 2024-09-16T08:51:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi160-75757-55"} : dispatch 2024-09-16T08:51:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:51:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-41"} : dispatch 2024-09-16T08:51:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:51:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi160-75757-55"}]': finished 2024-09-16T08:51:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-41"}]': finished 2024-09-16T08:51:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:04 smithi169 ceph-mon[33465]: osdmap e282: 8 total, 8 up, 8 in 2024-09-16T08:51:04.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[30137]: pgmap v369: 228 pgs: 228 active+clean; 8.4 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[30137]: osdmap e281: 8 total, 8 up, 8 in 2024-09-16T08:51:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi160-75757-55"} : dispatch 2024-09-16T08:51:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:51:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-41"} : dispatch 2024-09-16T08:51:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:51:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi160-75757-55"}]': finished 2024-09-16T08:51:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-41"}]': finished 2024-09-16T08:51:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[30137]: osdmap e282: 8 total, 8 up, 8 in 2024-09-16T08:51:04.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[40028]: pgmap v369: 228 pgs: 228 active+clean; 8.4 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:04.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[40028]: osdmap e281: 8 total, 8 up, 8 in 2024-09-16T08:51:04.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi160-75757-55"} : dispatch 2024-09-16T08:51:04.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:51:04.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-41"} : dispatch 2024-09-16T08:51:04.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:51:04.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1152546776' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi160-75757-55"}]': finished 2024-09-16T08:51:04.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-41"}]': finished 2024-09-16T08:51:04.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-mon[40028]: osdmap e282: 8 total, 8 up, 8 in 2024-09-16T08:51:05.349 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:51:04 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:51:05.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:05.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:05 smithi169 ceph-mon[33465]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:05.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:05.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:05 smithi169 ceph-mon[33465]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:05.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi160-75757-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:05.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:05 smithi169 ceph-mon[33465]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi160-75757-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:05.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:05.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:05 smithi160 ceph-mon[30137]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:05 smithi160 ceph-mon[30137]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi160-75757-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:05 smithi160 ceph-mon[30137]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi160-75757-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:05.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:51:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:51:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:51:05.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:05.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:05 smithi160 ceph-mon[40028]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:05.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:05.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:05 smithi160 ceph-mon[40028]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:05.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi160-75757-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:05.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:05 smithi160 ceph-mon[40028]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi160-75757-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:06.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:06 smithi169 ceph-mon[33465]: pgmap v372: 228 pgs: 228 active+clean; 8.4 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:06 smithi169 ceph-mon[33465]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi160-75757-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:06 smithi169 ceph-mon[33465]: osdmap e283: 8 total, 8 up, 8 in 2024-09-16T08:51:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:06 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi160-75757-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:06 smithi169 ceph-mon[33465]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi160-75757-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:06.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:06.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:06 smithi160 ceph-mon[30137]: pgmap v372: 228 pgs: 228 active+clean; 8.4 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:06.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:06 smithi160 ceph-mon[30137]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi160-75757-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:06 smithi160 ceph-mon[30137]: osdmap e283: 8 total, 8 up, 8 in 2024-09-16T08:51:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:06 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi160-75757-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:06 smithi160 ceph-mon[30137]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi160-75757-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:06 smithi160 ceph-mon[40028]: pgmap v372: 228 pgs: 228 active+clean; 8.4 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:06 smithi160 ceph-mon[40028]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi160-75757-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:06 smithi160 ceph-mon[40028]: osdmap e283: 8 total, 8 up, 8 in 2024-09-16T08:51:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:06 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi160-75757-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:06 smithi160 ceph-mon[40028]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi160-75757-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:07.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:07 smithi169 ceph-mon[33465]: osdmap e284: 8 total, 8 up, 8 in 2024-09-16T08:51:07.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:07 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:07.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:07 smithi160 ceph-mon[30137]: osdmap e284: 8 total, 8 up, 8 in 2024-09-16T08:51:07.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:07 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:07.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:07 smithi160 ceph-mon[40028]: osdmap e284: 8 total, 8 up, 8 in 2024-09-16T08:51:07.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:07 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:08.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:08 smithi169 ceph-mon[33465]: pgmap v375: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:08 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:08 smithi169 ceph-mon[33465]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi160-75757-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi160-75757-56"}]': finished 2024-09-16T08:51:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:51:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:08 smithi169 ceph-mon[33465]: osdmap e285: 8 total, 8 up, 8 in 2024-09-16T08:51:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi160-76079-6","var": "pg_num","format": "json"} : dispatch 2024-09-16T08:51:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:51:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:08 smithi160 ceph-mon[30137]: pgmap v375: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:08 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:08 smithi160 ceph-mon[30137]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi160-75757-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi160-75757-56"}]': finished 2024-09-16T08:51:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:51:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:08 smithi160 ceph-mon[30137]: osdmap e285: 8 total, 8 up, 8 in 2024-09-16T08:51:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi160-76079-6","var": "pg_num","format": "json"} : dispatch 2024-09-16T08:51:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:51:08.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:08 smithi160 ceph-mon[40028]: pgmap v375: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:08 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:08 smithi160 ceph-mon[40028]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi160-75757-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi160-75757-56"}]': finished 2024-09-16T08:51:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:51:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:08 smithi160 ceph-mon[40028]: osdmap e285: 8 total, 8 up, 8 in 2024-09-16T08:51:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi160-76079-6","var": "pg_num","format": "json"} : dispatch 2024-09-16T08:51:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:51:09.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:51:09.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:09 smithi169 ceph-mon[33465]: osdmap e286: 8 total, 8 up, 8 in 2024-09-16T08:51:09.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-16T08:51:09.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:51:09.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:09 smithi160 ceph-mon[30137]: osdmap e286: 8 total, 8 up, 8 in 2024-09-16T08:51:09.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-16T08:51:09.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:51:09.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:09 smithi160 ceph-mon[40028]: osdmap e286: 8 total, 8 up, 8 in 2024-09-16T08:51:09.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-16T08:51:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:10 smithi169 ceph-mon[33465]: pgmap v378: 236 pgs: 236 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_count","val": "8"}]': finished 2024-09-16T08:51:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:10 smithi169 ceph-mon[33465]: osdmap e287: 8 total, 8 up, 8 in 2024-09-16T08:51:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:51:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:10.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:10 smithi169 ceph-mon[33465]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:10.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:10 smithi160 ceph-mon[30137]: pgmap v378: 236 pgs: 236 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:10.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_count","val": "8"}]': finished 2024-09-16T08:51:10.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:10 smithi160 ceph-mon[30137]: osdmap e287: 8 total, 8 up, 8 in 2024-09-16T08:51:10.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:51:10.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:10.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:10 smithi160 ceph-mon[30137]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:10.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:10 smithi160 ceph-mon[40028]: pgmap v378: 236 pgs: 236 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:10.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_count","val": "8"}]': finished 2024-09-16T08:51:10.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:10 smithi160 ceph-mon[40028]: osdmap e287: 8 total, 8 up, 8 in 2024-09-16T08:51:10.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:51:10.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:10.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:10 smithi160 ceph-mon[40028]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:11.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-09-16T08:51:11.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-16T08:51:11.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: ok, hit_set contains 273:602f83fe:::foo:head 2024-09-16T08:51:11.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9039 ms) 2024-09-16T08:51:11.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-09-16T08:51:11.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg_num = 32 2024-09-16T08:51:11.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 0 ls 1726476672,0 2024-09-16T08:51:11.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 1 ls 1726476672,0 2024-09-16T08:51:11.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 2 ls 1726476672,0 2024-09-16T08:51:11.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 3 ls 1726476672,0 2024-09-16T08:51:11.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 4 ls 1726476672,0 2024-09-16T08:51:11.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 5 ls 1726476672,0 2024-09-16T08:51:11.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 6 ls 1726476672,0 2024-09-16T08:51:11.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 7 ls 1726476672,0 2024-09-16T08:51:11.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 8 ls 1726476672,0 2024-09-16T08:51:11.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 9 ls 1726476672,0 2024-09-16T08:51:11.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 10 ls 1726476672,0 2024-09-16T08:51:11.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 11 ls 1726476672,0 2024-09-16T08:51:11.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 12 ls 1726476672,0 2024-09-16T08:51:11.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 13 ls 1726476672,0 2024-09-16T08:51:11.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 14 ls 1726476672,0 2024-09-16T08:51:11.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 15 ls 1726476672,0 2024-09-16T08:51:11.487 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 16 ls 1726476672,0 2024-09-16T08:51:11.487 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 17 ls 1726476672,0 2024-09-16T08:51:11.487 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 18 ls 1726476672,0 2024-09-16T08:51:11.487 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 19 ls 1726476672,0 2024-09-16T08:51:11.487 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 20 ls 1726476672,0 2024-09-16T08:51:11.487 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 21 ls 1726476672,0 2024-09-16T08:51:11.487 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 22 ls 1726476672,0 2024-09-16T08:51:11.487 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 23 ls 1726476672,0 2024-09-16T08:51:11.487 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 24 ls 1726476672,0 2024-09-16T08:51:11.487 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 25 ls 1726476672,0 2024-09-16T08:51:11.487 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 26 ls 1726476672,0 2024-09-16T08:51:11.487 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 27 ls 1726476672,0 2024-09-16T08:51:11.488 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 28 ls 1726476672,0 2024-09-16T08:51:11.488 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 29 ls 1726476672,0 2024-09-16T08:51:11.488 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 30 ls 1726476672,0 2024-09-16T08:51:11.488 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 31 ls 1726476672,0 2024-09-16T08:51:11.488 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg_num = 32 2024-09-16T08:51:11.488 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:6cac518f:::0:head 2024-09-16T08:51:11.488 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:02547ec2:::1:head 2024-09-16T08:51:11.488 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:f905c69b:::2:head 2024-09-16T08:51:11.488 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:cfc208b3:::3:head 2024-09-16T08:51:11.488 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:d83876eb:::4:head 2024-09-16T08:51:11.488 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:b29083e3:::5:head 2024-09-16T08:51:11.488 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:c4fdafeb:::6:head 2024-09-16T08:51:11.489 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:5c6b0b28:::7:head 2024-09-16T08:51:11.489 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:bd63b0f1:::8:head 2024-09-16T08:51:11.489 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:e960b815:::9:head 2024-09-16T08:51:11.489 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:52ea6a34:::10:head 2024-09-16T08:51:11.489 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:89d3ae78:::11:head 2024-09-16T08:51:11.489 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:de5d7c5f:::12:head 2024-09-16T08:51:11.489 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:566253c9:::13:head 2024-09-16T08:51:11.489 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:62a1935d:::14:head 2024-09-16T08:51:11.489 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:863748b0:::15:head 2024-09-16T08:51:11.489 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:3958e169:::16:head 2024-09-16T08:51:11.489 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:4d4dabf9:::17:head 2024-09-16T08:51:11.489 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:8391935d:::18:head 2024-09-16T08:51:11.490 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:28883081:::19:head 2024-09-16T08:51:11.490 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:69259c59:::20:head 2024-09-16T08:51:11.490 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:4bdb80b7:::21:head 2024-09-16T08:51:11.490 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:a11c5d71:::22:head 2024-09-16T08:51:11.490 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:271af37b:::23:head 2024-09-16T08:51:11.490 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:95b121be:::24:head 2024-09-16T08:51:11.490 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:58d1031b:::25:head 2024-09-16T08:51:11.490 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:0a050783:::26:head 2024-09-16T08:51:11.490 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:c709704c:::27:head 2024-09-16T08:51:11.490 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:cbe56eaf:::28:head 2024-09-16T08:51:11.490 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:86b4b162:::29:head 2024-09-16T08:51:11.490 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:70d89383:::30:head 2024-09-16T08:51:11.490 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:dd450c7c:::31:head 2024-09-16T08:51:11.491 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:6d5729b1:::32:head 2024-09-16T08:51:11.491 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:c388f3fb:::33:head 2024-09-16T08:51:11.491 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:56cfea31:::34:head 2024-09-16T08:51:11.491 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:9dbc1bf7:::35:head 2024-09-16T08:51:11.491 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:40b74ccd:::36:head 2024-09-16T08:51:11.491 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:4d5aaf42:::37:head 2024-09-16T08:51:11.491 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:920f362c:::38:head 2024-09-16T08:51:11.491 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:6cc53222:::39:head 2024-09-16T08:51:11.491 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:9cad833f:::40:head 2024-09-16T08:51:11.491 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:1ea84d41:::41:head 2024-09-16T08:51:11.491 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:c4480ef6:::42:head 2024-09-16T08:51:11.491 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:a694361e:::43:head 2024-09-16T08:51:11.491 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:d1bd33e9:::44:head 2024-09-16T08:51:11.491 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:ddc2cd5d:::45:head 2024-09-16T08:51:11.492 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:2b782207:::46:head 2024-09-16T08:51:11.492 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:7b187fca:::47:head 2024-09-16T08:51:11.492 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:90ecdf6f:::48:head 2024-09-16T08:51:11.492 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:a5ed95fe:::49:head 2024-09-16T08:51:11.492 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:ea0eaa55:::50:head 2024-09-16T08:51:11.492 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:f33ef17b:::51:head 2024-09-16T08:51:11.492 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:a0d1b2f6:::52:head 2024-09-16T08:51:11.492 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:60c5229e:::53:head 2024-09-16T08:51:11.492 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:edcbc575:::54:head 2024-09-16T08:51:11.492 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:102cf253:::55:head 2024-09-16T08:51:11.492 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:efb7fb0b:::56:head 2024-09-16T08:51:11.492 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:50d0a326:::57:head 2024-09-16T08:51:11.492 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:d4dc5daf:::58:head 2024-09-16T08:51:11.492 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:3a130462:::59:head 2024-09-16T08:51:11.493 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:ec87ed71:::60:head 2024-09-16T08:51:11.493 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:d5bc9454:::61:head 2024-09-16T08:51:11.493 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:3ddfe313:::62:head 2024-09-16T08:51:11.493 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:7c2816b9:::63:head 2024-09-16T08:51:11.493 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:47e00e4d:::64:head 2024-09-16T08:51:11.493 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:c6410c18:::65:head 2024-09-16T08:51:11.493 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:b48ed237:::66:head 2024-09-16T08:51:11.493 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:cd63ad31:::67:head 2024-09-16T08:51:11.493 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:b179e92b:::68:head 2024-09-16T08:51:11.493 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:0d9f741a:::69:head 2024-09-16T08:51:11.493 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:6d3352ae:::70:head 2024-09-16T08:51:11.493 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:c6d5c19e:::71:head 2024-09-16T08:51:11.493 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:bc4729c3:::72:head 2024-09-16T08:51:11.493 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:77e930b9:::73:head 2024-09-16T08:51:11.494 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:0abeecfd:::74:head 2024-09-16T08:51:11.494 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:b7c37e15:::75:head 2024-09-16T08:51:11.494 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:b6378398:::76:head 2024-09-16T08:51:11.494 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:02bd68de:::77:head 2024-09-16T08:51:11.494 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:cc795d2d:::78:head 2024-09-16T08:51:11.494 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:630d4fea:::79:head 2024-09-16T08:51:11.494 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:e0d29ef5:::80:head 2024-09-16T08:51:11.494 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:fd6f13d2:::81:head 2024-09-16T08:51:11.494 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:606461d5:::82:head 2024-09-16T08:51:11.494 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:eadbdc43:::83:head 2024-09-16T08:51:11.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:8761d0bb:::84:head 2024-09-16T08:51:11.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:9ef0186f:::85:head 2024-09-16T08:51:11.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:e0d41294:::86:head 2024-09-16T08:51:11.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:961de695:::87:head 2024-09-16T08:51:11.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:1423148f:::88:head 2024-09-16T08:51:11.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:633a8fa2:::89:head 2024-09-16T08:51:11.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:a8653809:::90:head 2024-09-16T08:51:11.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:3dac8b33:::91:head 2024-09-16T08:51:11.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:35aad435:::92:head 2024-09-16T08:51:11.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:f6dcc343:::93:head 2024-09-16T08:51:11.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:dbbdad87:::94:head 2024-09-16T08:51:11.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:1cb48ce0:::95:head 2024-09-16T08:51:11.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:03cd461c:::96:head 2024-09-16T08:51:11.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:17a4ea99:::97:head 2024-09-16T08:51:11.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:9993c9a7:::98:head 2024-09-16T08:51:11.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:6394211c:::99:head 2024-09-16T08:51:11.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:94c7ae57:::100:head 2024-09-16T08:51:11.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:6fdee5bb:::101:head 2024-09-16T08:51:11.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:9a477fd1:::102:head 2024-09-16T08:51:11.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:eb850916:::103:head 2024-09-16T08:51:11.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:affc56b9:::104:head 2024-09-16T08:51:11.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:b42dc814:::105:head 2024-09-16T08:51:11.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:f319f8f0:::106:head 2024-09-16T08:51:11.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:9a40b9de:::107:head 2024-09-16T08:51:11.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:8b524f28:::108:head 2024-09-16T08:51:11.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:e3de589f:::109:head 2024-09-16T08:51:11.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:90f90a5b:::110:head 2024-09-16T08:51:11.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:a7b4f1d7:::111:head 2024-09-16T08:51:11.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:af51766e:::112:head 2024-09-16T08:51:11.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:b6f90bd1:::113:head 2024-09-16T08:51:11.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:e0261208:::114:head 2024-09-16T08:51:11.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:c9569ef7:::115:head 2024-09-16T08:51:11.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:61bebe50:::116:head 2024-09-16T08:51:11.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:fe93412b:::117:head 2024-09-16T08:51:11.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:d3d38bee:::118:head 2024-09-16T08:51:11.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:3100ba0c:::119:head 2024-09-16T08:51:11.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:d0560ada:::120:head 2024-09-16T08:51:11.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:f0ea8b35:::121:head 2024-09-16T08:51:11.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:766f231a:::122:head 2024-09-16T08:51:11.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:a07a2582:::123:head 2024-09-16T08:51:11.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:bd7c6b3a:::124:head 2024-09-16T08:51:11.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:fb2ddaff:::125:head 2024-09-16T08:51:11.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:4408e1fe:::126:head 2024-09-16T08:51:11.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:ee1df7a7:::127:head 2024-09-16T08:51:11.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:c3002909:::128:head 2024-09-16T08:51:11.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:4f48ffa9:::129:head 2024-09-16T08:51:11.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:edf38733:::130:head 2024-09-16T08:51:11.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:c08425c0:::131:head 2024-09-16T08:51:11.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:5f902d98:::132:head 2024-09-16T08:51:11.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:41ea2c93:::133:head 2024-09-16T08:51:11.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:813cee13:::134:head 2024-09-16T08:51:11.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:0131818d:::135:head 2024-09-16T08:51:11.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:26ba5a85:::136:head 2024-09-16T08:51:11.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:381b8a5a:::137:head 2024-09-16T08:51:11.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:28797e47:::138:head 2024-09-16T08:51:11.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:bfca7f22:::139:head 2024-09-16T08:51:11.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:36807075:::140:head 2024-09-16T08:51:11.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:80b03975:::141:head 2024-09-16T08:51:11.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:5c15709b:::142:head 2024-09-16T08:51:11.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:f39ea15e:::143:head 2024-09-16T08:51:11.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:ea992956:::144:head 2024-09-16T08:51:11.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:48887b1c:::145:head 2024-09-16T08:51:11.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:9f24a9dd:::146:head 2024-09-16T08:51:11.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:987f100b:::147:head 2024-09-16T08:51:11.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:d2dd3581:::148:head 2024-09-16T08:51:11.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:7fed1808:::149:head 2024-09-16T08:51:11.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:c80b70e9:::150:head 2024-09-16T08:51:11.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:85ed90f9:::151:head 2024-09-16T08:51:11.500 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:36428b24:::152:head 2024-09-16T08:51:11.500 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:d044c34a:::153:head 2024-09-16T08:51:11.500 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:7c18bf58:::154:head 2024-09-16T08:51:11.500 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:d1c21232:::155:head 2024-09-16T08:51:11.500 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:a7a3c575:::156:head 2024-09-16T08:51:11.500 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:87da0633:::157:head 2024-09-16T08:51:11.500 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:d5ac3822:::158:head 2024-09-16T08:51:11.500 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:3f20522d:::159:head 2024-09-16T08:51:11.500 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:6ca26563:::160:head 2024-09-16T08:51:11.500 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:532ce135:::161:head 2024-09-16T08:51:11.500 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:c78863e6:::162:head 2024-09-16T08:51:11.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:51:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:11 smithi169 ceph-mon[33465]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi160-75757-56"}]': finished 2024-09-16T08:51:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:11 smithi169 ceph-mon[33465]: osdmap e288: 8 total, 8 up, 8 in 2024-09-16T08:51:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-16T08:51:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:11 smithi169 ceph-mon[33465]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-16T08:51:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:11 smithi169 ceph-mon[33465]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi160-75757-56"}]': finished 2024-09-16T08:51:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:11 smithi169 ceph-mon[33465]: osdmap e289: 8 total, 8 up, 8 in 2024-09-16T08:51:11.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:51:11.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[30137]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi160-75757-56"}]': finished 2024-09-16T08:51:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[30137]: osdmap e288: 8 total, 8 up, 8 in 2024-09-16T08:51:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-16T08:51:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[30137]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-16T08:51:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[30137]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi160-75757-56"}]': finished 2024-09-16T08:51:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[30137]: osdmap e289: 8 total, 8 up, 8 in 2024-09-16T08:51:11.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:51:11.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[40028]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi160-75757-56"}]': finished 2024-09-16T08:51:11.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[40028]: osdmap e288: 8 total, 8 up, 8 in 2024-09-16T08:51:11.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/563988885' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:11.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-16T08:51:11.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[40028]: from='client.57179 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi160-75757-56"} : dispatch 2024-09-16T08:51:11.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-16T08:51:11.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[40028]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi160-75757-56"}]': finished 2024-09-16T08:51:11.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:11 smithi160 ceph-mon[40028]: osdmap e289: 8 total, 8 up, 8 in 2024-09-16T08:51:12.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:12 smithi169 ceph-mon[33465]: pgmap v381: 228 pgs: 228 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:12 smithi169 ceph-mon[33465]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:12 smithi169 ceph-mon[33465]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi160-75757-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:12 smithi169 ceph-mon[33465]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi160-75757-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi160-76079-43","var": "pg_num","format": "json"} : dispatch 2024-09-16T08:51:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:51:12.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-43"} : dispatch 2024-09-16T08:51:12.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[30137]: pgmap v381: 228 pgs: 228 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:12.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:12.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[30137]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:12.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:12.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[30137]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:12.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi160-75757-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:12.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[30137]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi160-75757-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:12.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi160-76079-43","var": "pg_num","format": "json"} : dispatch 2024-09-16T08:51:12.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:51:12.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-43"} : dispatch 2024-09-16T08:51:12.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[40028]: pgmap v381: 228 pgs: 228 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:12.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:12.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[40028]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:12.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:12.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[40028]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:12.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi160-75757-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:12.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[40028]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi160-75757-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:12.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi160-76079-43","var": "pg_num","format": "json"} : dispatch 2024-09-16T08:51:12.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:51:12.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-43"} : dispatch 2024-09-16T08:51:13.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:13 smithi169 ceph-mon[33465]: from='client.57185 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi160-75757-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:13.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-43"}]': finished 2024-09-16T08:51:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:13 smithi169 ceph-mon[33465]: osdmap e290: 8 total, 8 up, 8 in 2024-09-16T08:51:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi160-75757-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:13.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:13 smithi169 ceph-mon[33465]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi160-75757-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:13.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:13 smithi160 ceph-mon[30137]: from='client.57185 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi160-75757-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:13.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-43"}]': finished 2024-09-16T08:51:13.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:13 smithi160 ceph-mon[30137]: osdmap e290: 8 total, 8 up, 8 in 2024-09-16T08:51:13.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi160-75757-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:13.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:13 smithi160 ceph-mon[30137]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi160-75757-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:13.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:13 smithi160 ceph-mon[40028]: from='client.57185 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi160-75757-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:13.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-43"}]': finished 2024-09-16T08:51:13.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:13 smithi160 ceph-mon[40028]: osdmap e290: 8 total, 8 up, 8 in 2024-09-16T08:51:13.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi160-75757-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:13.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:13 smithi160 ceph-mon[40028]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi160-75757-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:14.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:14 smithi169 ceph-mon[33465]: pgmap v384: 228 pgs: 228 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:14 smithi169 ceph-mon[33465]: osdmap e291: 8 total, 8 up, 8 in 2024-09-16T08:51:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:14 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:14 smithi169 ceph-mon[33465]: from='client.57185 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi160-75757-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi160-75757-57"}]': finished 2024-09-16T08:51:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:14 smithi169 ceph-mon[33465]: osdmap e292: 8 total, 8 up, 8 in 2024-09-16T08:51:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:14 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:14.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:14 smithi160 ceph-mon[30137]: pgmap v384: 228 pgs: 228 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:14 smithi160 ceph-mon[30137]: osdmap e291: 8 total, 8 up, 8 in 2024-09-16T08:51:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:14 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:14 smithi160 ceph-mon[30137]: from='client.57185 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi160-75757-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi160-75757-57"}]': finished 2024-09-16T08:51:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:14 smithi160 ceph-mon[30137]: osdmap e292: 8 total, 8 up, 8 in 2024-09-16T08:51:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:14.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:14 smithi160 ceph-mon[40028]: pgmap v384: 228 pgs: 228 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:14.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:14 smithi160 ceph-mon[40028]: osdmap e291: 8 total, 8 up, 8 in 2024-09-16T08:51:14.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:14 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:14.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:14 smithi160 ceph-mon[40028]: from='client.57185 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi160-75757-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi160-75757-57"}]': finished 2024-09-16T08:51:14.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:14 smithi160 ceph-mon[40028]: osdmap e292: 8 total, 8 up, 8 in 2024-09-16T08:51:14.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:14 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:15.349 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:51:14 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:51:15.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:51:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:51:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:51:16.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:16 smithi169 ceph-mon[33465]: pgmap v387: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:51:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:16 smithi169 ceph-mon[33465]: osdmap e293: 8 total, 8 up, 8 in 2024-09-16T08:51:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:51:16.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:16.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:16 smithi160 ceph-mon[30137]: pgmap v387: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:51:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:16 smithi160 ceph-mon[30137]: osdmap e293: 8 total, 8 up, 8 in 2024-09-16T08:51:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:51:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:16 smithi160 ceph-mon[40028]: pgmap v387: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:51:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:16 smithi160 ceph-mon[40028]: osdmap e293: 8 total, 8 up, 8 in 2024-09-16T08:51:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:51:17.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:51:17.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:17 smithi169 ceph-mon[33465]: osdmap e294: 8 total, 8 up, 8 in 2024-09-16T08:51:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-16T08:51:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:17 smithi169 ceph-mon[33465]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_count","val": "3"}]': finished 2024-09-16T08:51:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:17 smithi169 ceph-mon[33465]: from='client.57185 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi160-75757-57"}]': finished 2024-09-16T08:51:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:17 smithi169 ceph-mon[33465]: osdmap e295: 8 total, 8 up, 8 in 2024-09-16T08:51:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-16T08:51:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:17 smithi169 ceph-mon[33465]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:17.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:51:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[40028]: osdmap e294: 8 total, 8 up, 8 in 2024-09-16T08:51:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-16T08:51:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[40028]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_count","val": "3"}]': finished 2024-09-16T08:51:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[40028]: from='client.57185 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi160-75757-57"}]': finished 2024-09-16T08:51:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[40028]: osdmap e295: 8 total, 8 up, 8 in 2024-09-16T08:51:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-16T08:51:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[40028]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:17.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:51:17.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[30137]: osdmap e294: 8 total, 8 up, 8 in 2024-09-16T08:51:17.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:17.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-16T08:51:17.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[30137]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:17.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_count","val": "3"}]': finished 2024-09-16T08:51:17.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[30137]: from='client.57185 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi160-75757-57"}]': finished 2024-09-16T08:51:17.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[30137]: osdmap e295: 8 total, 8 up, 8 in 2024-09-16T08:51:17.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3416498355' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:17.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-16T08:51:17.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:17 smithi160 ceph-mon[30137]: from='client.57185 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi160-75757-57"} : dispatch 2024-09-16T08:51:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:18 smithi169 ceph-mon[33465]: pgmap v390: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_period","val": "3"}]': finished 2024-09-16T08:51:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:18 smithi169 ceph-mon[33465]: from='client.57185 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi160-75757-57"}]': finished 2024-09-16T08:51:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:18 smithi169 ceph-mon[33465]: osdmap e296: 8 total, 8 up, 8 in 2024-09-16T08:51:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T08:51:18.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:18 smithi160 ceph-mon[30137]: pgmap v390: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:18.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_period","val": "3"}]': finished 2024-09-16T08:51:18.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:18 smithi160 ceph-mon[30137]: from='client.57185 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi160-75757-57"}]': finished 2024-09-16T08:51:18.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:18 smithi160 ceph-mon[30137]: osdmap e296: 8 total, 8 up, 8 in 2024-09-16T08:51:18.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T08:51:18.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:18 smithi160 ceph-mon[40028]: pgmap v390: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:18.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_period","val": "3"}]': finished 2024-09-16T08:51:18.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:18 smithi160 ceph-mon[40028]: from='client.57185 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi160-75757-57"}]': finished 2024-09-16T08:51:18.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:18 smithi160 ceph-mon[40028]: osdmap e296: 8 total, 8 up, 8 in 2024-09-16T08:51:18.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T08:51:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:19 smithi169 ceph-mon[33465]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:19 smithi169 ceph-mon[33465]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi160-75757-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:19 smithi169 ceph-mon[33465]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi160-75757-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:51:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T08:51:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:19 smithi169 ceph-mon[33465]: from='client.57191 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi160-75757-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:19 smithi169 ceph-mon[33465]: osdmap e297: 8 total, 8 up, 8 in 2024-09-16T08:51:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-16T08:51:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi160-75757-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:19 smithi169 ceph-mon[33465]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi160-75757-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[30137]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[30137]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi160-75757-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[30137]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi160-75757-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:51:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T08:51:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[30137]: from='client.57191 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi160-75757-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[30137]: osdmap e297: 8 total, 8 up, 8 in 2024-09-16T08:51:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-16T08:51:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi160-75757-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[30137]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi160-75757-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[40028]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[40028]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi160-75757-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:19.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[40028]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi160-75757-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:19.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:51:19.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T08:51:19.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[40028]: from='client.57191 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi160-75757-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:19.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[40028]: osdmap e297: 8 total, 8 up, 8 in 2024-09-16T08:51:19.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-16T08:51:19.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi160-75757-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:19.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:19 smithi160 ceph-mon[40028]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi160-75757-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:20.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:20 smithi169 ceph-mon[33465]: pgmap v393: 228 pgs: 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:20.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-16T08:51:20.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:20 smithi169 ceph-mon[33465]: osdmap e298: 8 total, 8 up, 8 in 2024-09-16T08:51:20.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:20 smithi160 ceph-mon[30137]: pgmap v393: 228 pgs: 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:20.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-16T08:51:20.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:20 smithi160 ceph-mon[30137]: osdmap e298: 8 total, 8 up, 8 in 2024-09-16T08:51:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:20 smithi160 ceph-mon[40028]: pgmap v393: 228 pgs: 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-16T08:51:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:20 smithi160 ceph-mon[40028]: osdmap e298: 8 total, 8 up, 8 in 2024-09-16T08:51:21.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:21 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:51:21.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:21 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:51:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:21 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:51:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:21 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:51:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:21 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:51:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:21 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:51:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:21 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:51:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:21 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:51:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:21 smithi169 ceph-mon[33465]: from='client.57191 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi160-75757-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi160-75757-58"}]': finished 2024-09-16T08:51:21.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:21 smithi169 ceph-mon[33465]: osdmap e299: 8 total, 8 up, 8 in 2024-09-16T08:51:21.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:51:21.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:51:21.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:51:21.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:51:21.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:51:21.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:51:21.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:51:21.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:51:21.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[30137]: from='client.57191 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi160-75757-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi160-75757-58"}]': finished 2024-09-16T08:51:21.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[30137]: osdmap e299: 8 total, 8 up, 8 in 2024-09-16T08:51:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:51:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:51:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:51:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:51:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:51:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:51:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:51:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:51:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[40028]: from='client.57191 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi160-75757-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi160-75757-58"}]': finished 2024-09-16T08:51:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:21 smithi160 ceph-mon[40028]: osdmap e299: 8 total, 8 up, 8 in 2024-09-16T08:51:22.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:22 smithi169 ceph-mon[33465]: pgmap v396: 228 pgs: 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:22.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:22 smithi169 ceph-mon[33465]: osdmap e300: 8 total, 8 up, 8 in 2024-09-16T08:51:22.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:22 smithi160 ceph-mon[30137]: pgmap v396: 228 pgs: 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:22.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:22 smithi160 ceph-mon[30137]: osdmap e300: 8 total, 8 up, 8 in 2024-09-16T08:51:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:22 smithi160 ceph-mon[40028]: pgmap v396: 228 pgs: 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:22 smithi160 ceph-mon[40028]: osdmap e300: 8 total, 8 up, 8 in 2024-09-16T08:51:24.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:24 smithi169 ceph-mon[33465]: pgmap v399: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:24 smithi169 ceph-mon[33465]: osdmap e301: 8 total, 8 up, 8 in 2024-09-16T08:51:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:24 smithi169 ceph-mon[33465]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:24 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:24 smithi160 ceph-mon[30137]: pgmap v399: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:24 smithi160 ceph-mon[30137]: osdmap e301: 8 total, 8 up, 8 in 2024-09-16T08:51:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:24 smithi160 ceph-mon[30137]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:24 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:24 smithi160 ceph-mon[40028]: pgmap v399: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:24 smithi160 ceph-mon[40028]: osdmap e301: 8 total, 8 up, 8 in 2024-09-16T08:51:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:24 smithi160 ceph-mon[40028]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:24 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:25.310 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:51:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:51:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:51:25.311 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:51:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:51:25.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:25 smithi169 ceph-mon[33465]: from='client.57191 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi160-75757-58"}]': finished 2024-09-16T08:51:25.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:25 smithi169 ceph-mon[33465]: osdmap e302: 8 total, 8 up, 8 in 2024-09-16T08:51:25.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:25.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:25 smithi169 ceph-mon[33465]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:25.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:25 smithi169 ceph-mon[33465]: from='client.57191 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi160-75757-58"}]': finished 2024-09-16T08:51:25.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:25 smithi169 ceph-mon[33465]: osdmap e303: 8 total, 8 up, 8 in 2024-09-16T08:51:25.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:25 smithi160 ceph-mon[40028]: from='client.57191 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi160-75757-58"}]': finished 2024-09-16T08:51:25.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:25 smithi160 ceph-mon[40028]: osdmap e302: 8 total, 8 up, 8 in 2024-09-16T08:51:25.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:25.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:25 smithi160 ceph-mon[40028]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:25.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:25 smithi160 ceph-mon[40028]: from='client.57191 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi160-75757-58"}]': finished 2024-09-16T08:51:25.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:25 smithi160 ceph-mon[40028]: osdmap e303: 8 total, 8 up, 8 in 2024-09-16T08:51:25.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:25 smithi160 ceph-mon[30137]: from='client.57191 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi160-75757-58"}]': finished 2024-09-16T08:51:25.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:25 smithi160 ceph-mon[30137]: osdmap e302: 8 total, 8 up, 8 in 2024-09-16T08:51:25.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3031298367' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:25.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:25 smithi160 ceph-mon[30137]: from='client.57191 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi160-75757-58"} : dispatch 2024-09-16T08:51:25.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:25 smithi160 ceph-mon[30137]: from='client.57191 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi160-75757-58"}]': finished 2024-09-16T08:51:25.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:25 smithi160 ceph-mon[30137]: osdmap e303: 8 total, 8 up, 8 in 2024-09-16T08:51:26.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:26 smithi169 ceph-mon[33465]: pgmap v402: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:51:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi160-75757-59"} : dispatch 2024-09-16T08:51:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi160-75757-59"} : dispatch 2024-09-16T08:51:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi160-75757-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:26.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:26.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:26 smithi160 ceph-mon[40028]: pgmap v402: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:51:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi160-75757-59"} : dispatch 2024-09-16T08:51:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi160-75757-59"} : dispatch 2024-09-16T08:51:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi160-75757-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:26 smithi160 ceph-mon[30137]: pgmap v402: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:51:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi160-75757-59"} : dispatch 2024-09-16T08:51:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi160-75757-59"} : dispatch 2024-09-16T08:51:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi160-75757-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi160-75757-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:27 smithi169 ceph-mon[33465]: osdmap e304: 8 total, 8 up, 8 in 2024-09-16T08:51:27.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi160-75757-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi160-75757-59"} : dispatch 2024-09-16T08:51:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi160-75757-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:27 smithi160 ceph-mon[30137]: osdmap e304: 8 total, 8 up, 8 in 2024-09-16T08:51:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi160-75757-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi160-75757-59"} : dispatch 2024-09-16T08:51:27.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi160-75757-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:27.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:27 smithi160 ceph-mon[40028]: osdmap e304: 8 total, 8 up, 8 in 2024-09-16T08:51:27.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi160-75757-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi160-75757-59"} : dispatch 2024-09-16T08:51:28.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:28 smithi169 ceph-mon[33465]: pgmap v405: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:51:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:28 smithi169 ceph-mon[33465]: osdmap e305: 8 total, 8 up, 8 in 2024-09-16T08:51:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi160-75757-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi160-75757-59"}]': finished 2024-09-16T08:51:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:28 smithi169 ceph-mon[33465]: osdmap e306: 8 total, 8 up, 8 in 2024-09-16T08:51:28.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:28 smithi160 ceph-mon[30137]: pgmap v405: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:51:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:28 smithi160 ceph-mon[30137]: osdmap e305: 8 total, 8 up, 8 in 2024-09-16T08:51:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi160-75757-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi160-75757-59"}]': finished 2024-09-16T08:51:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:28 smithi160 ceph-mon[30137]: osdmap e306: 8 total, 8 up, 8 in 2024-09-16T08:51:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:28 smithi160 ceph-mon[40028]: pgmap v405: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:51:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:28 smithi160 ceph-mon[40028]: osdmap e305: 8 total, 8 up, 8 in 2024-09-16T08:51:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi160-75757-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi160-75757-59"}]': finished 2024-09-16T08:51:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:28 smithi160 ceph-mon[40028]: osdmap e306: 8 total, 8 up, 8 in 2024-09-16T08:51:29.600 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:29 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:29.601 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:29 smithi160 ceph-mon[30137]: osdmap e307: 8 total, 8 up, 8 in 2024-09-16T08:51:29.601 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:29 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:29.601 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:29 smithi160 ceph-mon[40028]: osdmap e307: 8 total, 8 up, 8 in 2024-09-16T08:51:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:29 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:29.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:29 smithi169 ceph-mon[33465]: osdmap e307: 8 total, 8 up, 8 in 2024-09-16T08:51:30.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:30 smithi169 ceph-mon[33465]: pgmap v408: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-16T08:51:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:30 smithi169 ceph-mon[33465]: osdmap e308: 8 total, 8 up, 8 in 2024-09-16T08:51:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi160-75757-59"} : dispatch 2024-09-16T08:51:30.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:30 smithi160 ceph-mon[30137]: pgmap v408: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-16T08:51:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:30 smithi160 ceph-mon[30137]: osdmap e308: 8 total, 8 up, 8 in 2024-09-16T08:51:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi160-75757-59"} : dispatch 2024-09-16T08:51:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:30 smithi160 ceph-mon[40028]: pgmap v408: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-16T08:51:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:30 smithi160 ceph-mon[40028]: osdmap e308: 8 total, 8 up, 8 in 2024-09-16T08:51:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi160-75757-59"} : dispatch 2024-09-16T08:51:32.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:7462ddf6:::1.RoundTripAppendPP (2990 ms) 2024-09-16T08:51:32.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-09-16T08:51:32.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3000 ms) 2024-09-16T08:51:32.332 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-09-16T08:51:32.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3028 ms) 2024-09-16T08:51:32.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-09-16T08:51:32.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (2998 ms) 2024-09-16T08:51:32.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-09-16T08:51:32.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: setting pool EIO 2024-09-16T08:51:32.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: max_success 113, min_failed 114 2024-09-16T08:51:32.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4007 ms) 2024-09-16T08:51:32.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-09-16T08:51:32.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3019 ms) 2024-09-16T08:51:32.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (116677 ms total) 2024-09-16T08:51:32.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: 2024-09-16T08:51:32.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-09-16T08:51:32.333 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-09-16T08:51:32.334 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3002 ms) 2024-09-16T08:51:32.334 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-09-16T08:51:32.334 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9059 ms) 2024-09-16T08:51:32.334 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-09-16T08:51:32.334 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14918 ms) 2024-09-16T08:51:32.334 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-09-16T08:51:32.334 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3033 ms) 2024-09-16T08:51:32.334 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (30012 ms total) 2024-09-16T08:51:32.334 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: 2024-09-16T08:51:32.334 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-09-16T08:51:32.334 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-16T08:51:32.334 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3012 ms) 2024-09-16T08:51:32.334 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3012 ms total) 2024-09-16T08:51:32.334 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: 2024-09-16T08:51:32.335 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-09-16T08:51:32.335 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-09-16T08:51:32.335 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13971 ms) 2024-09-16T08:51:32.335 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-09-16T08:51:32.335 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6975 ms) 2024-09-16T08:51:32.335 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-09-16T08:51:32.335 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7049 ms) 2024-09-16T08:51:32.335 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-09-16T08:51:32.335 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7028 ms) 2024-09-16T08:51:32.335 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-09-16T08:51:32.335 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3021 ms) 2024-09-16T08:51:32.335 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-09-16T08:51:32.335 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7026 ms) 2024-09-16T08:51:32.335 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-09-16T08:51:32.336 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6886 ms) 2024-09-16T08:51:32.336 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-09-16T08:51:32.336 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7021 ms) 2024-09-16T08:51:32.336 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-09-16T08:51:32.336 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7007 ms) 2024-09-16T08:51:32.336 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-09-16T08:51:32.336 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7060 ms) 2024-09-16T08:51:32.336 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-09-16T08:51:32.336 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7030 ms) 2024-09-16T08:51:32.336 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-09-16T08:51:32.336 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7013 ms) 2024-09-16T08:51:32.336 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-09-16T08:51:32.336 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7027 ms) 2024-09-16T08:51:32.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:32 smithi169 ceph-mon[33465]: pgmap v411: 228 pgs: 228 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-16T08:51:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi160-75757-59"}]': finished 2024-09-16T08:51:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:32 smithi169 ceph-mon[33465]: osdmap e309: 8 total, 8 up, 8 in 2024-09-16T08:51:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi160-75757-59"} : dispatch 2024-09-16T08:51:32.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:32 smithi160 ceph-mon[30137]: pgmap v411: 228 pgs: 228 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-16T08:51:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi160-75757-59"}]': finished 2024-09-16T08:51:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:32 smithi160 ceph-mon[30137]: osdmap e309: 8 total, 8 up, 8 in 2024-09-16T08:51:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi160-75757-59"} : dispatch 2024-09-16T08:51:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:32 smithi160 ceph-mon[40028]: pgmap v411: 228 pgs: 228 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-16T08:51:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi160-75757-59"}]': finished 2024-09-16T08:51:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:32 smithi160 ceph-mon[40028]: osdmap e309: 8 total, 8 up, 8 in 2024-09-16T08:51:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi160-75757-59"} : dispatch 2024-09-16T08:51:33.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi160-75757-59"}]': finished 2024-09-16T08:51:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:33 smithi169 ceph-mon[33465]: osdmap e310: 8 total, 8 up, 8 in 2024-09-16T08:51:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:51:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-45"} : dispatch 2024-09-16T08:51:33.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi160-75757-59"}]': finished 2024-09-16T08:51:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:33 smithi160 ceph-mon[30137]: osdmap e310: 8 total, 8 up, 8 in 2024-09-16T08:51:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:51:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-45"} : dispatch 2024-09-16T08:51:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3526670421' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi160-75757-59"}]': finished 2024-09-16T08:51:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:33 smithi160 ceph-mon[40028]: osdmap e310: 8 total, 8 up, 8 in 2024-09-16T08:51:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:51:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-45"} : dispatch 2024-09-16T08:51:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:34 smithi160 ceph-mon[30137]: pgmap v414: 228 pgs: 228 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-45"}]': finished 2024-09-16T08:51:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:34 smithi160 ceph-mon[30137]: osdmap e311: 8 total, 8 up, 8 in 2024-09-16T08:51:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/4248162314' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi160-75757-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:34 smithi160 ceph-mon[30137]: from='client.48370 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi160-75757-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:34.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:51:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:34 smithi160 ceph-mon[40028]: pgmap v414: 228 pgs: 228 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-45"}]': finished 2024-09-16T08:51:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:34 smithi160 ceph-mon[40028]: osdmap e311: 8 total, 8 up, 8 in 2024-09-16T08:51:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/4248162314' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi160-75757-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:34.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:34 smithi160 ceph-mon[40028]: from='client.48370 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi160-75757-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:34.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:51:34.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:34 smithi169 ceph-mon[33465]: pgmap v414: 228 pgs: 228 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:34.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-45"}]': finished 2024-09-16T08:51:34.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:34 smithi169 ceph-mon[33465]: osdmap e311: 8 total, 8 up, 8 in 2024-09-16T08:51:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/4248162314' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi160-75757-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:34 smithi169 ceph-mon[33465]: from='client.48370 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi160-75757-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:51:35.349 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:51:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:51:35.375 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:51:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:51:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:51:35.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:35 smithi160 ceph-mon[30137]: from='client.48370 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi160-75757-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:51:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:35 smithi160 ceph-mon[30137]: osdmap e312: 8 total, 8 up, 8 in 2024-09-16T08:51:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:35 smithi160 ceph-mon[30137]: osdmap e313: 8 total, 8 up, 8 in 2024-09-16T08:51:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:35 smithi160 ceph-mon[40028]: from='client.48370 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi160-75757-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:51:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:35 smithi160 ceph-mon[40028]: osdmap e312: 8 total, 8 up, 8 in 2024-09-16T08:51:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:35 smithi160 ceph-mon[40028]: osdmap e313: 8 total, 8 up, 8 in 2024-09-16T08:51:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:35.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:35 smithi169 ceph-mon[33465]: from='client.48370 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi160-75757-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:51:35.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:35 smithi169 ceph-mon[33465]: osdmap e312: 8 total, 8 up, 8 in 2024-09-16T08:51:35.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:35 smithi169 ceph-mon[33465]: osdmap e313: 8 total, 8 up, 8 in 2024-09-16T08:51:35.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:36.755 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[30137]: pgmap v417: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:36.755 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.755 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[30137]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.755 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.755 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[30137]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.755 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi160-75757-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:36.755 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[30137]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi160-75757-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:36.755 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:51:36.756 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[30137]: from='client.57209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi160-75757-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:36.756 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[30137]: osdmap e314: 8 total, 8 up, 8 in 2024-09-16T08:51:36.756 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi160-75757-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.756 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[30137]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi160-75757-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.756 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:36.756 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[40028]: pgmap v417: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:36.756 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.756 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[40028]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.756 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.757 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[40028]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.757 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi160-75757-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:36.757 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[40028]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi160-75757-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:36.757 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:51:36.757 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[40028]: from='client.57209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi160-75757-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:36.757 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[40028]: osdmap e314: 8 total, 8 up, 8 in 2024-09-16T08:51:36.757 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi160-75757-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.757 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:36 smithi160 ceph-mon[40028]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi160-75757-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:36 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:36 smithi169 ceph-mon[33465]: pgmap v417: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:36 smithi169 ceph-mon[33465]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:36 smithi169 ceph-mon[33465]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi160-75757-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:36.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:36 smithi169 ceph-mon[33465]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi160-75757-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:36.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:51:36.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:36 smithi169 ceph-mon[33465]: from='client.57209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi160-75757-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:36.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:36 smithi169 ceph-mon[33465]: osdmap e314: 8 total, 8 up, 8 in 2024-09-16T08:51:36.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi160-75757-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:36.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:36 smithi169 ceph-mon[33465]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi160-75757-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:51:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:51:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:37 smithi160 ceph-mon[30137]: osdmap e315: 8 total, 8 up, 8 in 2024-09-16T08:51:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-47"} : dispatch 2024-09-16T08:51:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:51:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:51:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:37 smithi160 ceph-mon[40028]: osdmap e315: 8 total, 8 up, 8 in 2024-09-16T08:51:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-47"} : dispatch 2024-09-16T08:51:37.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:51:37.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:51:37.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:37 smithi169 ceph-mon[33465]: osdmap e315: 8 total, 8 up, 8 in 2024-09-16T08:51:37.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-47"} : dispatch 2024-09-16T08:51:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:38 smithi160 ceph-mon[30137]: pgmap v420: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:38 smithi160 ceph-mon[30137]: from='client.57209 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi160-75757-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi160-75757-61"}]': finished 2024-09-16T08:51:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-47"}]': finished 2024-09-16T08:51:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:38 smithi160 ceph-mon[30137]: osdmap e316: 8 total, 8 up, 8 in 2024-09-16T08:51:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-47", "mode": "writeback"} : dispatch 2024-09-16T08:51:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:38 smithi160 ceph-mon[40028]: pgmap v420: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:38 smithi160 ceph-mon[40028]: from='client.57209 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi160-75757-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi160-75757-61"}]': finished 2024-09-16T08:51:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-47"}]': finished 2024-09-16T08:51:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:38 smithi160 ceph-mon[40028]: osdmap e316: 8 total, 8 up, 8 in 2024-09-16T08:51:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-47", "mode": "writeback"} : dispatch 2024-09-16T08:51:38.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:38 smithi169 ceph-mon[33465]: pgmap v420: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:38 smithi169 ceph-mon[33465]: from='client.57209 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi160-75757-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi160-75757-61"}]': finished 2024-09-16T08:51:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-47"}]': finished 2024-09-16T08:51:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:38 smithi169 ceph-mon[33465]: osdmap e316: 8 total, 8 up, 8 in 2024-09-16T08:51:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-47", "mode": "writeback"} : dispatch 2024-09-16T08:51:39.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:39 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:51:39.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-47", "mode": "writeback"}]': finished 2024-09-16T08:51:39.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:39 smithi160 ceph-mon[30137]: osdmap e317: 8 total, 8 up, 8 in 2024-09-16T08:51:39.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:51:39.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:39 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:51:39.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-47", "mode": "writeback"}]': finished 2024-09-16T08:51:39.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:39 smithi160 ceph-mon[40028]: osdmap e317: 8 total, 8 up, 8 in 2024-09-16T08:51:39.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:51:39.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:39 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:51:39.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-47", "mode": "writeback"}]': finished 2024-09-16T08:51:39.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:39 smithi169 ceph-mon[33465]: osdmap e317: 8 total, 8 up, 8 in 2024-09-16T08:51:39.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:51:40.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:40 smithi160 ceph-mon[30137]: pgmap v423: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-16T08:51:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:40 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_count","val": "2"}]': finished 2024-09-16T08:51:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:40 smithi160 ceph-mon[30137]: osdmap e318: 8 total, 8 up, 8 in 2024-09-16T08:51:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:51:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:40 smithi160 ceph-mon[30137]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:40 smithi160 ceph-mon[40028]: pgmap v423: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-16T08:51:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:40 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_count","val": "2"}]': finished 2024-09-16T08:51:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:40 smithi160 ceph-mon[40028]: osdmap e318: 8 total, 8 up, 8 in 2024-09-16T08:51:40.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:51:40.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:40.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:40 smithi160 ceph-mon[40028]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:40.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:40 smithi169 ceph-mon[33465]: pgmap v423: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-16T08:51:40.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:40 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_count","val": "2"}]': finished 2024-09-16T08:51:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:40 smithi169 ceph-mon[33465]: osdmap e318: 8 total, 8 up, 8 in 2024-09-16T08:51:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:51:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:40 smithi169 ceph-mon[33465]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:42.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:42 smithi160 ceph-mon[30137]: pgmap v426: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-16T08:51:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:51:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:42 smithi160 ceph-mon[30137]: from='client.57209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi160-75757-61"}]': finished 2024-09-16T08:51:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:42 smithi160 ceph-mon[30137]: osdmap e319: 8 total, 8 up, 8 in 2024-09-16T08:51:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T08:51:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:42 smithi160 ceph-mon[30137]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:42 smithi160 ceph-mon[40028]: pgmap v426: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-16T08:51:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:51:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:42 smithi160 ceph-mon[40028]: from='client.57209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi160-75757-61"}]': finished 2024-09-16T08:51:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:42 smithi160 ceph-mon[40028]: osdmap e319: 8 total, 8 up, 8 in 2024-09-16T08:51:42.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T08:51:42.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:42.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:42 smithi160 ceph-mon[40028]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:42.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:42 smithi169 ceph-mon[33465]: pgmap v426: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-16T08:51:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:51:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:42 smithi169 ceph-mon[33465]: from='client.57209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi160-75757-61"}]': finished 2024-09-16T08:51:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:42 smithi169 ceph-mon[33465]: osdmap e319: 8 total, 8 up, 8 in 2024-09-16T08:51:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T08:51:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/76422987' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:42 smithi169 ceph-mon[33465]: from='client.57209 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi160-75757-61"} : dispatch 2024-09-16T08:51:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:51:43.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T08:51:43.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: from='client.57209 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi160-75757-61"}]': finished 2024-09-16T08:51:43.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: osdmap e320: 8 total, 8 up, 8 in 2024-09-16T08:51:43.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T08:51:43.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:43.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:43.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-16T08:51:43.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: from='client.57215 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:43.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: osdmap e321: 8 total, 8 up, 8 in 2024-09-16T08:51:43.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-16T08:51:43.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi160-75757-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[30137]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi160-75757-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:51:43.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T08:51:43.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: from='client.57209 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi160-75757-61"}]': finished 2024-09-16T08:51:43.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: osdmap e320: 8 total, 8 up, 8 in 2024-09-16T08:51:43.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T08:51:43.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:43.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:43.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-16T08:51:43.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: from='client.57215 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:43.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: osdmap e321: 8 total, 8 up, 8 in 2024-09-16T08:51:43.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-16T08:51:43.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi160-75757-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:43 smithi160 ceph-mon[40028]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi160-75757-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:51:43.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T08:51:43.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: from='client.57209 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi160-75757-61"}]': finished 2024-09-16T08:51:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: osdmap e320: 8 total, 8 up, 8 in 2024-09-16T08:51:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T08:51:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-16T08:51:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: from='client.57215 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:43.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: osdmap e321: 8 total, 8 up, 8 in 2024-09-16T08:51:43.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-16T08:51:43.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi160-75757-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:43.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:43 smithi169 ceph-mon[33465]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi160-75757-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:44.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:44 smithi160 ceph-mon[40028]: pgmap v429: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:44.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-16T08:51:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:44 smithi160 ceph-mon[30137]: pgmap v429: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-16T08:51:44.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:44 smithi169 ceph-mon[33465]: pgmap v429: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:44.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-16T08:51:45.349 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:51:45 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:51:45.386 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:51:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:51:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:51:45.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:45 smithi160 ceph-mon[30137]: osdmap e322: 8 total, 8 up, 8 in 2024-09-16T08:51:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-16T08:51:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:45 smithi160 ceph-mon[30137]: from='client.57215 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi160-75757-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi160-75757-62"}]': finished 2024-09-16T08:51:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-16T08:51:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:45 smithi160 ceph-mon[30137]: osdmap e323: 8 total, 8 up, 8 in 2024-09-16T08:51:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:45 smithi160 ceph-mon[40028]: osdmap e322: 8 total, 8 up, 8 in 2024-09-16T08:51:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-16T08:51:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:45 smithi160 ceph-mon[40028]: from='client.57215 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi160-75757-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi160-75757-62"}]': finished 2024-09-16T08:51:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-16T08:51:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:45 smithi160 ceph-mon[40028]: osdmap e323: 8 total, 8 up, 8 in 2024-09-16T08:51:45.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:45 smithi169 ceph-mon[33465]: osdmap e322: 8 total, 8 up, 8 in 2024-09-16T08:51:45.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-16T08:51:45.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:45 smithi169 ceph-mon[33465]: from='client.57215 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi160-75757-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi160-75757-62"}]': finished 2024-09-16T08:51:45.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-16T08:51:45.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:45 smithi169 ceph-mon[33465]: osdmap e323: 8 total, 8 up, 8 in 2024-09-16T08:51:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:46 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:46 smithi160 ceph-mon[30137]: pgmap v432: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:51:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:51:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:46 smithi160 ceph-mon[30137]: osdmap e324: 8 total, 8 up, 8 in 2024-09-16T08:51:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-47"} : dispatch 2024-09-16T08:51:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:46 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:46 smithi160 ceph-mon[40028]: pgmap v432: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:51:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:51:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:46 smithi160 ceph-mon[40028]: osdmap e324: 8 total, 8 up, 8 in 2024-09-16T08:51:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-47"} : dispatch 2024-09-16T08:51:46.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:46 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:46.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:46 smithi169 ceph-mon[33465]: pgmap v432: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:46.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:51:46.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:51:46.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:46 smithi169 ceph-mon[33465]: osdmap e324: 8 total, 8 up, 8 in 2024-09-16T08:51:46.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-47"} : dispatch 2024-09-16T08:51:48.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:48 smithi160 ceph-mon[30137]: pgmap v435: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:48.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-47"}]': finished 2024-09-16T08:51:48.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:48 smithi160 ceph-mon[30137]: osdmap e325: 8 total, 8 up, 8 in 2024-09-16T08:51:48.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:48.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:48 smithi160 ceph-mon[30137]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:48.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:48 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:48.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:51:48.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-47"} : dispatch 2024-09-16T08:51:48.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:48 smithi160 ceph-mon[40028]: pgmap v435: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:48.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-47"}]': finished 2024-09-16T08:51:48.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:48 smithi160 ceph-mon[40028]: osdmap e325: 8 total, 8 up, 8 in 2024-09-16T08:51:48.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:48.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:48 smithi160 ceph-mon[40028]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:48.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:48 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:48.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:51:48.703 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-47"} : dispatch 2024-09-16T08:51:48.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:48 smithi169 ceph-mon[33465]: pgmap v435: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:51:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-47"}]': finished 2024-09-16T08:51:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:48 smithi169 ceph-mon[33465]: osdmap e325: 8 total, 8 up, 8 in 2024-09-16T08:51:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:48 smithi169 ceph-mon[33465]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:48 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:51:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-47"} : dispatch 2024-09-16T08:51:49.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:49 smithi160 ceph-mon[30137]: from='client.57215 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62"}]': finished 2024-09-16T08:51:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:49 smithi160 ceph-mon[30137]: osdmap e326: 8 total, 8 up, 8 in 2024-09-16T08:51:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:49 smithi160 ceph-mon[30137]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:51:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:49 smithi160 ceph-mon[30137]: from='client.57215 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi160-75757-62"}]': finished 2024-09-16T08:51:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:49 smithi160 ceph-mon[30137]: osdmap e327: 8 total, 8 up, 8 in 2024-09-16T08:51:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:49 smithi160 ceph-mon[40028]: from='client.57215 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62"}]': finished 2024-09-16T08:51:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:49 smithi160 ceph-mon[40028]: osdmap e326: 8 total, 8 up, 8 in 2024-09-16T08:51:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:49 smithi160 ceph-mon[40028]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:51:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:49 smithi160 ceph-mon[40028]: from='client.57215 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi160-75757-62"}]': finished 2024-09-16T08:51:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:49 smithi160 ceph-mon[40028]: osdmap e327: 8 total, 8 up, 8 in 2024-09-16T08:51:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:49.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:49 smithi169 ceph-mon[33465]: from='client.57215 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi160-75757-62"}]': finished 2024-09-16T08:51:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:49 smithi169 ceph-mon[33465]: osdmap e326: 8 total, 8 up, 8 in 2024-09-16T08:51:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1848318495' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:49 smithi169 ceph-mon[33465]: from='client.57215 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi160-75757-62"} : dispatch 2024-09-16T08:51:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:51:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:49 smithi169 ceph-mon[33465]: from='client.57215 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi160-75757-62"}]': finished 2024-09-16T08:51:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:49 smithi169 ceph-mon[33465]: osdmap e327: 8 total, 8 up, 8 in 2024-09-16T08:51:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:51:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:50 smithi160 ceph-mon[30137]: pgmap v438: 196 pgs: 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T08:51:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi160-75757-63"} : dispatch 2024-09-16T08:51:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi160-75757-63"} : dispatch 2024-09-16T08:51:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi160-75757-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:51:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi160-75757-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:50 smithi160 ceph-mon[30137]: osdmap e328: 8 total, 8 up, 8 in 2024-09-16T08:51:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemovePP_smithi160-75757-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi160-75757-63"} : dispatch 2024-09-16T08:51:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:50 smithi160 ceph-mon[40028]: pgmap v438: 196 pgs: 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T08:51:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi160-75757-63"} : dispatch 2024-09-16T08:51:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi160-75757-63"} : dispatch 2024-09-16T08:51:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi160-75757-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:51:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi160-75757-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:50 smithi160 ceph-mon[40028]: osdmap e328: 8 total, 8 up, 8 in 2024-09-16T08:51:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemovePP_smithi160-75757-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi160-75757-63"} : dispatch 2024-09-16T08:51:50.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:50 smithi169 ceph-mon[33465]: pgmap v438: 196 pgs: 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T08:51:50.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi160-75757-63"} : dispatch 2024-09-16T08:51:50.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi160-75757-63"} : dispatch 2024-09-16T08:51:50.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi160-75757-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:50.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:51:50.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi160-75757-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:50.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:50 smithi169 ceph-mon[33465]: osdmap e328: 8 total, 8 up, 8 in 2024-09-16T08:51:50.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemovePP_smithi160-75757-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi160-75757-63"} : dispatch 2024-09-16T08:51:51.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:51:51.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:51:51.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:51 smithi160 ceph-mon[30137]: osdmap e329: 8 total, 8 up, 8 in 2024-09-16T08:51:51.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-49"} : dispatch 2024-09-16T08:51:51.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:51:51.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:51:51.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:51 smithi160 ceph-mon[40028]: osdmap e329: 8 total, 8 up, 8 in 2024-09-16T08:51:51.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-49"} : dispatch 2024-09-16T08:51:51.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:51:51.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:51:51.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:51 smithi169 ceph-mon[33465]: osdmap e329: 8 total, 8 up, 8 in 2024-09-16T08:51:51.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-49"} : dispatch 2024-09-16T08:51:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:52 smithi160 ceph-mon[40028]: pgmap v441: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T08:51:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi160-75757-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi160-75757-63"}]': finished 2024-09-16T08:51:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-49"}]': finished 2024-09-16T08:51:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:52 smithi160 ceph-mon[40028]: osdmap e330: 8 total, 8 up, 8 in 2024-09-16T08:51:52.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-49", "mode": "readproxy"} : dispatch 2024-09-16T08:51:52.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:52 smithi160 ceph-mon[30137]: pgmap v441: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T08:51:52.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi160-75757-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi160-75757-63"}]': finished 2024-09-16T08:51:52.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-49"}]': finished 2024-09-16T08:51:52.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:52 smithi160 ceph-mon[30137]: osdmap e330: 8 total, 8 up, 8 in 2024-09-16T08:51:52.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-49", "mode": "readproxy"} : dispatch 2024-09-16T08:51:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:52 smithi169 ceph-mon[33465]: pgmap v441: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T08:51:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi160-75757-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi160-75757-63"}]': finished 2024-09-16T08:51:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-49"}]': finished 2024-09-16T08:51:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:52 smithi169 ceph-mon[33465]: osdmap e330: 8 total, 8 up, 8 in 2024-09-16T08:51:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-49", "mode": "readproxy"} : dispatch 2024-09-16T08:51:53.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:53 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:51:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-49", "mode": "readproxy"}]': finished 2024-09-16T08:51:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:53 smithi160 ceph-mon[40028]: osdmap e331: 8 total, 8 up, 8 in 2024-09-16T08:51:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:53 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:51:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-49", "mode": "readproxy"}]': finished 2024-09-16T08:51:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:53 smithi160 ceph-mon[30137]: osdmap e331: 8 total, 8 up, 8 in 2024-09-16T08:51:53.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:53 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:51:53.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-49", "mode": "readproxy"}]': finished 2024-09-16T08:51:53.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:53 smithi169 ceph-mon[33465]: osdmap e331: 8 total, 8 up, 8 in 2024-09-16T08:51:54.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:54 smithi160 ceph-mon[30137]: pgmap v444: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:54.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:54 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:54.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:54 smithi160 ceph-mon[30137]: osdmap e332: 8 total, 8 up, 8 in 2024-09-16T08:51:54.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi160-75757-63"} : dispatch 2024-09-16T08:51:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:54 smithi160 ceph-mon[40028]: pgmap v444: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:54 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:54 smithi160 ceph-mon[40028]: osdmap e332: 8 total, 8 up, 8 in 2024-09-16T08:51:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi160-75757-63"} : dispatch 2024-09-16T08:51:54.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:54 smithi169 ceph-mon[33465]: pgmap v444: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:51:54.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:54 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:54.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:54 smithi169 ceph-mon[33465]: osdmap e332: 8 total, 8 up, 8 in 2024-09-16T08:51:54.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi160-75757-63"} : dispatch 2024-09-16T08:51:55.349 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:51:55 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:51:55.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:51:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:51:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:51:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:56 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:56 smithi160 ceph-mon[30137]: pgmap v447: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:51:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi160-75757-63"}]': finished 2024-09-16T08:51:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:56 smithi160 ceph-mon[30137]: osdmap e333: 8 total, 8 up, 8 in 2024-09-16T08:51:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi160-75757-63"} : dispatch 2024-09-16T08:51:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:56 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:56 smithi160 ceph-mon[40028]: pgmap v447: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:51:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi160-75757-63"}]': finished 2024-09-16T08:51:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:56 smithi160 ceph-mon[40028]: osdmap e333: 8 total, 8 up, 8 in 2024-09-16T08:51:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi160-75757-63"} : dispatch 2024-09-16T08:51:56.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:56 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:51:56.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:56 smithi169 ceph-mon[33465]: pgmap v447: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:51:56.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi160-75757-63"}]': finished 2024-09-16T08:51:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:56 smithi169 ceph-mon[33465]: osdmap e333: 8 total, 8 up, 8 in 2024-09-16T08:51:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi160-75757-63"} : dispatch 2024-09-16T08:51:57.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi160-75757-63"}]': finished 2024-09-16T08:51:57.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:57 smithi160 ceph-mon[30137]: osdmap e334: 8 total, 8 up, 8 in 2024-09-16T08:51:57.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:57.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:57 smithi160 ceph-mon[30137]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:57.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:57.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:57 smithi160 ceph-mon[30137]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:57.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi160-75757-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:57.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:57 smithi160 ceph-mon[30137]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi160-75757-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:57 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi160-75757-63"}]': finished 2024-09-16T08:51:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:57 smithi160 ceph-mon[40028]: osdmap e334: 8 total, 8 up, 8 in 2024-09-16T08:51:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:57 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:57 smithi160 ceph-mon[40028]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:57 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:57 smithi160 ceph-mon[40028]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:57 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi160-75757-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:57 smithi160 ceph-mon[40028]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi160-75757-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:57.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:57 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3633861634' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi160-75757-63"}]': finished 2024-09-16T08:51:57.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:57 smithi169 ceph-mon[33465]: osdmap e334: 8 total, 8 up, 8 in 2024-09-16T08:51:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:57 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:57 smithi169 ceph-mon[33465]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:57 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:57 smithi169 ceph-mon[33465]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:57 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi160-75757-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:57 smithi169 ceph-mon[33465]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi160-75757-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:51:58.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:58 smithi160 ceph-mon[40028]: pgmap v450: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:51:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:58 smithi160 ceph-mon[40028]: from='client.57227 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi160-75757-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:58 smithi160 ceph-mon[40028]: osdmap e335: 8 total, 8 up, 8 in 2024-09-16T08:51:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi160-75757-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:58 smithi160 ceph-mon[40028]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi160-75757-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:58.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:58 smithi160 ceph-mon[30137]: pgmap v450: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:51:58.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:58 smithi160 ceph-mon[30137]: from='client.57227 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi160-75757-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:58.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:58 smithi160 ceph-mon[30137]: osdmap e335: 8 total, 8 up, 8 in 2024-09-16T08:51:58.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi160-75757-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:58.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:58 smithi160 ceph-mon[30137]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi160-75757-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:58.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:58 smithi169 ceph-mon[33465]: pgmap v450: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:51:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:58 smithi169 ceph-mon[33465]: from='client.57227 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi160-75757-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:51:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:58 smithi169 ceph-mon[33465]: osdmap e335: 8 total, 8 up, 8 in 2024-09-16T08:51:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi160-75757-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:58 smithi169 ceph-mon[33465]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi160-75757-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:51:59.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:59 smithi160 ceph-mon[30137]: osdmap e336: 8 total, 8 up, 8 in 2024-09-16T08:51:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:59 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:59 smithi160 ceph-mon[30137]: from='client.57227 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi160-75757-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi160-75757-64"}]': finished 2024-09-16T08:51:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:51:59 smithi160 ceph-mon[30137]: osdmap e337: 8 total, 8 up, 8 in 2024-09-16T08:51:59.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:59 smithi160 ceph-mon[40028]: osdmap e336: 8 total, 8 up, 8 in 2024-09-16T08:51:59.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:59 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:59.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:59 smithi160 ceph-mon[40028]: from='client.57227 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi160-75757-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi160-75757-64"}]': finished 2024-09-16T08:51:59.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:51:59 smithi160 ceph-mon[40028]: osdmap e337: 8 total, 8 up, 8 in 2024-09-16T08:51:59.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:59 smithi169 ceph-mon[33465]: osdmap e336: 8 total, 8 up, 8 in 2024-09-16T08:51:59.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:59 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:51:59.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:59 smithi169 ceph-mon[33465]: from='client.57227 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi160-75757-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi160-75757-64"}]': finished 2024-09-16T08:51:59.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:51:59 smithi169 ceph-mon[33465]: osdmap e337: 8 total, 8 up, 8 in 2024-09-16T08:52:00.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:00 smithi160 ceph-mon[30137]: pgmap v453: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:52:00.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:00 smithi160 ceph-mon[30137]: osdmap e338: 8 total, 8 up, 8 in 2024-09-16T08:52:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:00 smithi160 ceph-mon[40028]: pgmap v453: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:52:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:00 smithi160 ceph-mon[40028]: osdmap e338: 8 total, 8 up, 8 in 2024-09-16T08:52:00.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:00 smithi169 ceph-mon[33465]: pgmap v453: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:52:00.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:00 smithi169 ceph-mon[33465]: osdmap e338: 8 total, 8 up, 8 in 2024-09-16T08:52:02.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:02 smithi160 ceph-mon[30137]: pgmap v456: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:52:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:02 smithi160 ceph-mon[30137]: osdmap e339: 8 total, 8 up, 8 in 2024-09-16T08:52:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:02 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:52:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:02 smithi160 ceph-mon[30137]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:52:02.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:02 smithi160 ceph-mon[40028]: pgmap v456: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:52:02.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:02 smithi160 ceph-mon[40028]: osdmap e339: 8 total, 8 up, 8 in 2024-09-16T08:52:02.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:02 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:52:02.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:02 smithi160 ceph-mon[40028]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:52:02.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:02 smithi169 ceph-mon[33465]: pgmap v456: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:52:02.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:02 smithi169 ceph-mon[33465]: osdmap e339: 8 total, 8 up, 8 in 2024-09-16T08:52:02.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:02 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:52:02.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:02 smithi169 ceph-mon[33465]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:52:03.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:03 smithi160 ceph-mon[30137]: from='client.57227 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi160-75757-64"}]': finished 2024-09-16T08:52:03.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:03 smithi160 ceph-mon[30137]: osdmap e340: 8 total, 8 up, 8 in 2024-09-16T08:52:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:52:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:03 smithi160 ceph-mon[30137]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:52:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:03 smithi160 ceph-mon[40028]: from='client.57227 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi160-75757-64"}]': finished 2024-09-16T08:52:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:03 smithi160 ceph-mon[40028]: osdmap e340: 8 total, 8 up, 8 in 2024-09-16T08:52:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:52:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:03 smithi160 ceph-mon[40028]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:52:03.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:03 smithi169 ceph-mon[33465]: from='client.57227 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi160-75757-64"}]': finished 2024-09-16T08:52:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:03 smithi169 ceph-mon[33465]: osdmap e340: 8 total, 8 up, 8 in 2024-09-16T08:52:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3973072340' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:52:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:03 smithi169 ceph-mon[33465]: from='client.57227 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi160-75757-64"} : dispatch 2024-09-16T08:52:04.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[30137]: pgmap v459: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:52:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[30137]: from='client.57227 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi160-75757-64"}]': finished 2024-09-16T08:52:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[30137]: osdmap e341: 8 total, 8 up, 8 in 2024-09-16T08:52:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi160-75757-65"} : dispatch 2024-09-16T08:52:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi160-75757-65"} : dispatch 2024-09-16T08:52:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi160-75757-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:52:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:52:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:52:04.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[40028]: pgmap v459: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:52:04.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[40028]: from='client.57227 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi160-75757-64"}]': finished 2024-09-16T08:52:04.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[40028]: osdmap e341: 8 total, 8 up, 8 in 2024-09-16T08:52:04.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi160-75757-65"} : dispatch 2024-09-16T08:52:04.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi160-75757-65"} : dispatch 2024-09-16T08:52:04.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi160-75757-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:52:04.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:04.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:52:04.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:52:04.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:04 smithi169 ceph-mon[33465]: pgmap v459: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:52:04.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:04 smithi169 ceph-mon[33465]: from='client.57227 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi160-75757-64"}]': finished 2024-09-16T08:52:04.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:04 smithi169 ceph-mon[33465]: osdmap e341: 8 total, 8 up, 8 in 2024-09-16T08:52:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi160-75757-65"} : dispatch 2024-09-16T08:52:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi160-75757-65"} : dispatch 2024-09-16T08:52:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi160-75757-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:52:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:52:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:52:05.349 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:52:05 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:52:05.439 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:52:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:52:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:52:05.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi160-75757-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:52:05.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:52:05.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:05 smithi160 ceph-mon[30137]: osdmap e342: 8 total, 8 up, 8 in 2024-09-16T08:52:05.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-49"} : dispatch 2024-09-16T08:52:05.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi160-75757-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi160-75757-65"} : dispatch 2024-09-16T08:52:05.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi160-75757-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:52:05.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:52:05.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:05 smithi160 ceph-mon[40028]: osdmap e342: 8 total, 8 up, 8 in 2024-09-16T08:52:05.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-49"} : dispatch 2024-09-16T08:52:05.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi160-75757-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi160-75757-65"} : dispatch 2024-09-16T08:52:05.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi160-75757-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:52:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:52:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:05 smithi169 ceph-mon[33465]: osdmap e342: 8 total, 8 up, 8 in 2024-09-16T08:52:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-49"} : dispatch 2024-09-16T08:52:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi160-75757-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi160-75757-65"} : dispatch 2024-09-16T08:52:06.452 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP263:head 2024-09-16T08:52:06.452 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:5d165639:::164:head 2024-09-16T08:52:06.452 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:f43765fc:::165:head 2024-09-16T08:52:06.452 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:b4c720e9:::166:head 2024-09-16T08:52:06.452 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:e694b040:::167:head 2024-09-16T08:52:06.452 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:afa38db2:::168:head 2024-09-16T08:52:06.453 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:77ba9f53:::169:head 2024-09-16T08:52:06.453 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:87495034:::170:head 2024-09-16T08:52:06.453 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:7c96bf0e:::171:head 2024-09-16T08:52:06.453 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:dbe346cc:::172:head 2024-09-16T08:52:06.453 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:e943ec24:::173:head 2024-09-16T08:52:06.453 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:f97a9c0c:::174:head 2024-09-16T08:52:06.453 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:6f26e74d:::175:head 2024-09-16T08:52:06.453 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:4f95e106:::176:head 2024-09-16T08:52:06.453 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:0e6f2f8f:::177:head 2024-09-16T08:52:06.453 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:05db05f1:::178:head 2024-09-16T08:52:06.453 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:38a78d66:::179:head 2024-09-16T08:52:06.453 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:d095610b:::180:head 2024-09-16T08:52:06.453 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:a1a9d709:::181:head 2024-09-16T08:52:06.454 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:1e5d39db:::182:head 2024-09-16T08:52:06.454 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:f7df4fb9:::183:head 2024-09-16T08:52:06.454 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:03a7f161:::184:head 2024-09-16T08:52:06.454 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:ba70721e:::185:head 2024-09-16T08:52:06.454 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:28e5662d:::186:head 2024-09-16T08:52:06.454 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:973d52de:::187:head 2024-09-16T08:52:06.454 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:4303eb1c:::188:head 2024-09-16T08:52:06.454 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:b990b48e:::189:head 2024-09-16T08:52:06.454 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:29b8165b:::190:head 2024-09-16T08:52:06.454 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:3547f197:::191:head 2024-09-16T08:52:06.454 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:7e260936:::192:head 2024-09-16T08:52:06.454 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:1abec7b1:::193:head 2024-09-16T08:52:06.454 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:10fdda93:::194:head 2024-09-16T08:52:06.454 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:15817eea:::195:head 2024-09-16T08:52:06.455 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:770bab57:::196:head 2024-09-16T08:52:06.455 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:ed9e13e7:::197:head 2024-09-16T08:52:06.455 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:71471a8f:::198:head 2024-09-16T08:52:06.455 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 275:10fb1d02:::199:head 2024-09-16T08:52:06.455 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8066 ms) 2024-09-16T08:52:06.455 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-09-16T08:52:06.455 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726476681,0 2024-09-16T08:52:06.455 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: first is 1726476681 2024-09-16T08:52:06.455 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726476681,0 2024-09-16T08:52:06.455 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726476681,0 2024-09-16T08:52:06.455 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726476681,1726476683,1726476684,0 2024-09-16T08:52:06.455 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726476681,1726476683,1726476684,0 2024-09-16T08:52:06.455 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726476681,1726476683,1726476684,0 2024-09-16T08:52:06.455 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726476681,1726476683,1726476684,1726476686,1726476687,0 2024-09-16T08:52:06.456 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726476681,1726476683,1726476684,1726476686,1726476687,0 2024-09-16T08:52:06.456 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726476681,1726476683,1726476684,1726476686,1726476687,0 2024-09-16T08:52:06.456 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726476681,1726476683,1726476684,1726476686,1726476687,1726476689,1726476690,0 2024-09-16T08:52:06.456 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726476681,1726476683,1726476684,1726476686,1726476687,1726476689,1726476690,0 2024-09-16T08:52:06.456 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726476681,1726476683,1726476684,1726476686,1726476687,1726476689,1726476690,0 2024-09-16T08:52:06.456 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726476684,1726476686,1726476687,1726476689,1726476690,1726476692,1726476693,0 2024-09-16T08:52:06.456 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: first now 1726476684, trimmed 2024-09-16T08:52:06.456 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (21063 ms) 2024-09-16T08:52:06.456 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-09-16T08:52:06.456 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: foo0 2024-09-16T08:52:06.456 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-16T08:52:06.456 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14024 ms) 2024-09-16T08:52:06.456 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-09-16T08:52:06.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:06.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:06 smithi160 ceph-mon[30137]: pgmap v462: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:52:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:06 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:52:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:06 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-49"}]': finished 2024-09-16T08:52:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:06 smithi160 ceph-mon[30137]: osdmap e343: 8 total, 8 up, 8 in 2024-09-16T08:52:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:06 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:06 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-49"} : dispatch 2024-09-16T08:52:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:06 smithi160 ceph-mon[40028]: pgmap v462: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:52:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:06 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:52:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:06 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-49"}]': finished 2024-09-16T08:52:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:06 smithi160 ceph-mon[40028]: osdmap e343: 8 total, 8 up, 8 in 2024-09-16T08:52:06.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:06 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:06.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:06 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-49"} : dispatch 2024-09-16T08:52:06.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:06.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:06 smithi169 ceph-mon[33465]: pgmap v462: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:52:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:06 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:52:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:06 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-49"}]': finished 2024-09-16T08:52:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:06 smithi169 ceph-mon[33465]: osdmap e343: 8 total, 8 up, 8 in 2024-09-16T08:52:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:06 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:06 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-49"} : dispatch 2024-09-16T08:52:07.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:07 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi160-75757-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi160-75757-65"}]': finished 2024-09-16T08:52:07.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:07 smithi169 ceph-mon[33465]: osdmap e344: 8 total, 8 up, 8 in 2024-09-16T08:52:07.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:07 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi160-75757-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi160-75757-65"}]': finished 2024-09-16T08:52:07.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:07 smithi160 ceph-mon[30137]: osdmap e344: 8 total, 8 up, 8 in 2024-09-16T08:52:07.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:07 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi160-75757-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi160-75757-65"}]': finished 2024-09-16T08:52:07.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:07 smithi160 ceph-mon[40028]: osdmap e344: 8 total, 8 up, 8 in 2024-09-16T08:52:08.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:08 smithi169 ceph-mon[33465]: pgmap v465: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:52:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:08 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:08 smithi169 ceph-mon[33465]: osdmap e345: 8 total, 8 up, 8 in 2024-09-16T08:52:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:08.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:08 smithi160 ceph-mon[30137]: pgmap v465: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:52:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:08 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:08 smithi160 ceph-mon[30137]: osdmap e345: 8 total, 8 up, 8 in 2024-09-16T08:52:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:08 smithi160 ceph-mon[40028]: pgmap v465: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:52:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:08 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:08 smithi160 ceph-mon[40028]: osdmap e345: 8 total, 8 up, 8 in 2024-09-16T08:52:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:09.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:09.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:09 smithi169 ceph-mon[33465]: osdmap e346: 8 total, 8 up, 8 in 2024-09-16T08:52:09.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi160-75757-65"} : dispatch 2024-09-16T08:52:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:52:09.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:09.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:09 smithi160 ceph-mon[30137]: osdmap e346: 8 total, 8 up, 8 in 2024-09-16T08:52:09.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi160-75757-65"} : dispatch 2024-09-16T08:52:09.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:52:09.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:09.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:09 smithi160 ceph-mon[40028]: osdmap e346: 8 total, 8 up, 8 in 2024-09-16T08:52:09.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi160-75757-65"} : dispatch 2024-09-16T08:52:09.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:52:10.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:10 smithi169 ceph-mon[33465]: pgmap v468: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:52:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi160-75757-65"}]': finished 2024-09-16T08:52:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:52:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:10 smithi169 ceph-mon[33465]: osdmap e347: 8 total, 8 up, 8 in 2024-09-16T08:52:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi160-75757-65"} : dispatch 2024-09-16T08:52:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-51"} : dispatch 2024-09-16T08:52:10.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:10 smithi160 ceph-mon[30137]: pgmap v468: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:52:10.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi160-75757-65"}]': finished 2024-09-16T08:52:10.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:52:10.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:10 smithi160 ceph-mon[30137]: osdmap e347: 8 total, 8 up, 8 in 2024-09-16T08:52:10.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi160-75757-65"} : dispatch 2024-09-16T08:52:10.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-51"} : dispatch 2024-09-16T08:52:10.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:10 smithi160 ceph-mon[40028]: pgmap v468: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:52:10.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi160-75757-65"}]': finished 2024-09-16T08:52:10.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:52:10.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:10 smithi160 ceph-mon[40028]: osdmap e347: 8 total, 8 up, 8 in 2024-09-16T08:52:10.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi160-75757-65"} : dispatch 2024-09-16T08:52:10.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-51"} : dispatch 2024-09-16T08:52:11.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi160-75757-65"}]': finished 2024-09-16T08:52:11.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-51"}]': finished 2024-09-16T08:52:11.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:11 smithi169 ceph-mon[33465]: osdmap e348: 8 total, 8 up, 8 in 2024-09-16T08:52:11.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-51", "mode": "writeback"} : dispatch 2024-09-16T08:52:11.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:11 smithi169 ceph-mon[33465]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:11 smithi169 ceph-mon[33465]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi160-75757-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:52:11.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:11 smithi169 ceph-mon[33465]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi160-75757-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:52:11.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:11 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:52:11.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-51", "mode": "writeback"}]': finished 2024-09-16T08:52:11.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:11 smithi169 ceph-mon[33465]: from='client.57239 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi160-75757-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:52:11.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:11 smithi169 ceph-mon[33465]: osdmap e349: 8 total, 8 up, 8 in 2024-09-16T08:52:11.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi160-75757-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:11 smithi169 ceph-mon[33465]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi160-75757-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi160-75757-65"}]': finished 2024-09-16T08:52:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-51"}]': finished 2024-09-16T08:52:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[30137]: osdmap e348: 8 total, 8 up, 8 in 2024-09-16T08:52:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-51", "mode": "writeback"} : dispatch 2024-09-16T08:52:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[30137]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[30137]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi160-75757-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:52:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[30137]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi160-75757-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:52:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:52:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-51", "mode": "writeback"}]': finished 2024-09-16T08:52:11.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[30137]: from='client.57239 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi160-75757-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:52:11.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[30137]: osdmap e349: 8 total, 8 up, 8 in 2024-09-16T08:52:11.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi160-75757-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[30137]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi160-75757-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2697943035' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi160-75757-65"}]': finished 2024-09-16T08:52:11.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-51"}]': finished 2024-09-16T08:52:11.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[40028]: osdmap e348: 8 total, 8 up, 8 in 2024-09-16T08:52:11.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-51", "mode": "writeback"} : dispatch 2024-09-16T08:52:11.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[40028]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[40028]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi160-75757-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:52:11.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[40028]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi160-75757-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:52:11.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:52:11.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-51", "mode": "writeback"}]': finished 2024-09-16T08:52:11.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[40028]: from='client.57239 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi160-75757-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:52:11.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[40028]: osdmap e349: 8 total, 8 up, 8 in 2024-09-16T08:52:11.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi160-75757-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:11.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:11 smithi160 ceph-mon[40028]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi160-75757-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:12.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:12 smithi169 ceph-mon[33465]: pgmap v471: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:52:12.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:52:12.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_count","val": "2"}]': finished 2024-09-16T08:52:12.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:12 smithi169 ceph-mon[33465]: osdmap e350: 8 total, 8 up, 8 in 2024-09-16T08:52:12.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:52:12.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:12 smithi160 ceph-mon[30137]: pgmap v471: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:52:12.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:52:12.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_count","val": "2"}]': finished 2024-09-16T08:52:12.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:12 smithi160 ceph-mon[30137]: osdmap e350: 8 total, 8 up, 8 in 2024-09-16T08:52:12.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:52:12.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:12 smithi160 ceph-mon[40028]: pgmap v471: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:52:12.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:52:12.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_count","val": "2"}]': finished 2024-09-16T08:52:12.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:12 smithi160 ceph-mon[40028]: osdmap e350: 8 total, 8 up, 8 in 2024-09-16T08:52:12.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:52:14.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:14 smithi169 ceph-mon[33465]: pgmap v474: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:52:14.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:14 smithi169 ceph-mon[33465]: from='client.57239 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi160-75757-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi160-75757-66"}]': finished 2024-09-16T08:52:14.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:14 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:52:14.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:14 smithi169 ceph-mon[33465]: osdmap e351: 8 total, 8 up, 8 in 2024-09-16T08:52:14.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:14 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T08:52:14.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:14 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:14.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:14 smithi160 ceph-mon[30137]: pgmap v474: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:52:14.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:14 smithi160 ceph-mon[30137]: from='client.57239 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi160-75757-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi160-75757-66"}]': finished 2024-09-16T08:52:14.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:52:14.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:14 smithi160 ceph-mon[30137]: osdmap e351: 8 total, 8 up, 8 in 2024-09-16T08:52:14.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T08:52:14.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:14 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:14.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:14 smithi160 ceph-mon[40028]: pgmap v474: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:52:14.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:14 smithi160 ceph-mon[40028]: from='client.57239 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi160-75757-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi160-75757-66"}]': finished 2024-09-16T08:52:14.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:14 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:52:14.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:14 smithi160 ceph-mon[40028]: osdmap e351: 8 total, 8 up, 8 in 2024-09-16T08:52:14.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:14 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T08:52:14.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:14 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:15.349 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:52:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:52:15.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:52:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:52:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:52:15.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:15 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:52:15.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:15 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T08:52:15.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:15 smithi169 ceph-mon[33465]: osdmap e352: 8 total, 8 up, 8 in 2024-09-16T08:52:15.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:15 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T08:52:15.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:15 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-16T08:52:15.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:15 smithi169 ceph-mon[33465]: osdmap e353: 8 total, 8 up, 8 in 2024-09-16T08:52:15.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:15 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:52:15.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:15 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T08:52:15.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:15 smithi160 ceph-mon[30137]: osdmap e352: 8 total, 8 up, 8 in 2024-09-16T08:52:15.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:15 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T08:52:15.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:15 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-16T08:52:15.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:15 smithi160 ceph-mon[30137]: osdmap e353: 8 total, 8 up, 8 in 2024-09-16T08:52:15.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:15 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:52:15.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:15 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T08:52:15.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:15 smithi160 ceph-mon[40028]: osdmap e352: 8 total, 8 up, 8 in 2024-09-16T08:52:15.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:15 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T08:52:15.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:15 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-16T08:52:15.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:15 smithi160 ceph-mon[40028]: osdmap e353: 8 total, 8 up, 8 in 2024-09-16T08:52:16.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:16.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:16 smithi169 ceph-mon[33465]: pgmap v477: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-16T08:52:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-16T08:52:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:16 smithi169 ceph-mon[33465]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "target_max_objects","val": "1"}]': finished 2024-09-16T08:52:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:16 smithi169 ceph-mon[33465]: from='client.57239 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi160-75757-66"}]': finished 2024-09-16T08:52:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:16 smithi169 ceph-mon[33465]: osdmap e354: 8 total, 8 up, 8 in 2024-09-16T08:52:16.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:16 smithi160 ceph-mon[30137]: pgmap v477: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-16T08:52:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-16T08:52:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:16 smithi160 ceph-mon[30137]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "target_max_objects","val": "1"}]': finished 2024-09-16T08:52:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:16 smithi160 ceph-mon[30137]: from='client.57239 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi160-75757-66"}]': finished 2024-09-16T08:52:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:16 smithi160 ceph-mon[30137]: osdmap e354: 8 total, 8 up, 8 in 2024-09-16T08:52:16.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:16.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:16 smithi160 ceph-mon[40028]: pgmap v477: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-16T08:52:16.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:16.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-16T08:52:16.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:16 smithi160 ceph-mon[40028]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:16.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-51","var": "target_max_objects","val": "1"}]': finished 2024-09-16T08:52:16.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:16 smithi160 ceph-mon[40028]: from='client.57239 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi160-75757-66"}]': finished 2024-09-16T08:52:16.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:16 smithi160 ceph-mon[40028]: osdmap e354: 8 total, 8 up, 8 in 2024-09-16T08:52:17.494 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ 2024-09-16T08:52:17.494 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3023 ms) 2024-09-16T08:52:17.494 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-09-16T08:52:17.494 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7021 ms) 2024-09-16T08:52:17.494 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-09-16T08:52:17.494 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7003 ms) 2024-09-16T08:52:17.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-09-16T08:52:17.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7017 ms) 2024-09-16T08:52:17.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-09-16T08:52:17.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7031 ms) 2024-09-16T08:52:17.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-09-16T08:52:17.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7051 ms) 2024-09-16T08:52:17.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-09-16T08:52:17.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7011 ms) 2024-09-16T08:52:17.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139272 ms total) 2024-09-16T08:52:17.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: 2024-09-16T08:52:17.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-09-16T08:52:17.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (288973 ms total) 2024-09-16T08:52:17.495 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-09-16T08:52:17.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:17.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:17 smithi169 ceph-mon[33465]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:17.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:17 smithi169 ceph-mon[33465]: from='client.57239 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi160-75757-66"}]': finished 2024-09-16T08:52:17.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:17 smithi169 ceph-mon[33465]: osdmap e355: 8 total, 8 up, 8 in 2024-09-16T08:52:17.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:17.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:17 smithi160 ceph-mon[30137]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:17.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:17 smithi160 ceph-mon[30137]: from='client.57239 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi160-75757-66"}]': finished 2024-09-16T08:52:17.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:17 smithi160 ceph-mon[30137]: osdmap e355: 8 total, 8 up, 8 in 2024-09-16T08:52:17.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1782572736' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:17.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:17 smithi160 ceph-mon[40028]: from='client.57239 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi160-75757-66"} : dispatch 2024-09-16T08:52:17.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:17 smithi160 ceph-mon[40028]: from='client.57239 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi160-75757-66"}]': finished 2024-09-16T08:52:17.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:17 smithi160 ceph-mon[40028]: osdmap e355: 8 total, 8 up, 8 in 2024-09-16T08:52:18.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:18 smithi169 ceph-mon[33465]: pgmap v480: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-16T08:52:18.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:18 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-16T08:52:18.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:18 smithi160 ceph-mon[30137]: pgmap v480: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-16T08:52:18.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:18 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-16T08:52:18.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:18 smithi160 ceph-mon[40028]: pgmap v480: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-16T08:52:18.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:18 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-16T08:52:20.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:52:20.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:52:20.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:19 smithi169 ceph-mon[33465]: pgmap v482: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:52:20.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:52:20.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:52:20.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:19 smithi160 ceph-mon[40028]: pgmap v482: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:52:20.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:52:20.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:52:20.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:19 smithi160 ceph-mon[30137]: pgmap v482: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:52:22.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:22 smithi169 ceph-mon[33465]: pgmap v483: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:52:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:22 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:52:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:22 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:52:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:22 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:52:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:22 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:52:22.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:22 smithi160 ceph-mon[40028]: pgmap v483: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:52:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:22 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:52:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:22 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:52:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:22 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:52:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:22 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:52:22.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:22 smithi160 ceph-mon[30137]: pgmap v483: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:52:22.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:22 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:52:22.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:22 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:52:22.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:22 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:52:22.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:22 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:52:24.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:24 smithi169 ceph-mon[33465]: pgmap v484: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 667 B/s rd, 0 B/s wr, 0 op/s 2024-09-16T08:52:24.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:24 smithi160 ceph-mon[30137]: pgmap v484: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 667 B/s rd, 0 B/s wr, 0 op/s 2024-09-16T08:52:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:24 smithi160 ceph-mon[40028]: pgmap v484: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 667 B/s rd, 0 B/s wr, 0 op/s 2024-09-16T08:52:25.349 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:52:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:52:25.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:52:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:52:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:52:26.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:26 smithi169 ceph-mon[33465]: pgmap v485: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:52:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:26 smithi160 ceph-mon[30137]: pgmap v485: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:52:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:26 smithi160 ceph-mon[40028]: pgmap v485: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:52:27.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:27.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:27.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:28.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:28 smithi169 ceph-mon[33465]: pgmap v486: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:52:28.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:52:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:28 smithi169 ceph-mon[33465]: osdmap e356: 8 total, 8 up, 8 in 2024-09-16T08:52:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-51"} : dispatch 2024-09-16T08:52:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:28 smithi160 ceph-mon[30137]: pgmap v486: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:52:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:52:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:28 smithi160 ceph-mon[30137]: osdmap e356: 8 total, 8 up, 8 in 2024-09-16T08:52:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-51"} : dispatch 2024-09-16T08:52:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:28 smithi160 ceph-mon[40028]: pgmap v486: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:52:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:52:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:28 smithi160 ceph-mon[40028]: osdmap e356: 8 total, 8 up, 8 in 2024-09-16T08:52:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-51"} : dispatch 2024-09-16T08:52:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-51"}]': finished 2024-09-16T08:52:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:29 smithi160 ceph-mon[30137]: osdmap e357: 8 total, 8 up, 8 in 2024-09-16T08:52:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:29.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-51"} : dispatch 2024-09-16T08:52:29.599 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-51"}]': finished 2024-09-16T08:52:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:29 smithi160 ceph-mon[40028]: osdmap e357: 8 total, 8 up, 8 in 2024-09-16T08:52:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:29.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-51"} : dispatch 2024-09-16T08:52:29.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-51"}]': finished 2024-09-16T08:52:29.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:29 smithi169 ceph-mon[33465]: osdmap e357: 8 total, 8 up, 8 in 2024-09-16T08:52:29.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:29.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-51"} : dispatch 2024-09-16T08:52:30.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:30 smithi169 ceph-mon[33465]: pgmap v489: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:52:30.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:30 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-16T08:52:30.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:30 smithi169 ceph-mon[33465]: osdmap e358: 8 total, 8 up, 8 in 2024-09-16T08:52:30.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:30 smithi160 ceph-mon[40028]: pgmap v489: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:52:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:30 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-16T08:52:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:30 smithi160 ceph-mon[40028]: osdmap e358: 8 total, 8 up, 8 in 2024-09-16T08:52:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:30 smithi160 ceph-mon[30137]: pgmap v489: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:52:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:30 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-16T08:52:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:30 smithi160 ceph-mon[30137]: osdmap e358: 8 total, 8 up, 8 in 2024-09-16T08:52:31.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:31 smithi169 ceph-mon[33465]: osdmap e359: 8 total, 8 up, 8 in 2024-09-16T08:52:31.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:31.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:31 smithi160 ceph-mon[40028]: osdmap e359: 8 total, 8 up, 8 in 2024-09-16T08:52:31.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:31 smithi160 ceph-mon[30137]: osdmap e359: 8 total, 8 up, 8 in 2024-09-16T08:52:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:32.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:32 smithi169 ceph-mon[33465]: pgmap v492: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-16T08:52:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:32 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:32 smithi169 ceph-mon[33465]: osdmap e360: 8 total, 8 up, 8 in 2024-09-16T08:52:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-53"} : dispatch 2024-09-16T08:52:32.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:32 smithi160 ceph-mon[40028]: pgmap v492: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-16T08:52:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:32 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:32 smithi160 ceph-mon[40028]: osdmap e360: 8 total, 8 up, 8 in 2024-09-16T08:52:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-53"} : dispatch 2024-09-16T08:52:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:32 smithi160 ceph-mon[30137]: pgmap v492: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-16T08:52:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:32 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:32 smithi160 ceph-mon[30137]: osdmap e360: 8 total, 8 up, 8 in 2024-09-16T08:52:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:32.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-53"} : dispatch 2024-09-16T08:52:33.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:33 smithi169 ceph-mon[33465]: osdmap e361: 8 total, 8 up, 8 in 2024-09-16T08:52:33.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:33 smithi160 ceph-mon[30137]: osdmap e361: 8 total, 8 up, 8 in 2024-09-16T08:52:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:33 smithi160 ceph-mon[40028]: osdmap e361: 8 total, 8 up, 8 in 2024-09-16T08:52:34.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:34 smithi169 ceph-mon[33465]: pgmap v495: 196 pgs: 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:52:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:34 smithi169 ceph-mon[33465]: osdmap e362: 8 total, 8 up, 8 in 2024-09-16T08:52:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:52:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:52:34.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:34 smithi160 ceph-mon[30137]: pgmap v495: 196 pgs: 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:52:34.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:34 smithi160 ceph-mon[30137]: osdmap e362: 8 total, 8 up, 8 in 2024-09-16T08:52:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:52:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:52:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:34 smithi160 ceph-mon[40028]: pgmap v495: 196 pgs: 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:52:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:34 smithi160 ceph-mon[40028]: osdmap e362: 8 total, 8 up, 8 in 2024-09-16T08:52:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:52:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:52:35.297 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:52:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:52:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:52:35.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:35.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:35 smithi169 ceph-mon[33465]: osdmap e363: 8 total, 8 up, 8 in 2024-09-16T08:52:35.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:52:35.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:35.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-55"} : dispatch 2024-09-16T08:52:35.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:35 smithi169 ceph-mon[33465]: osdmap e364: 8 total, 8 up, 8 in 2024-09-16T08:52:35.350 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:52:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:52:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:35 smithi160 ceph-mon[30137]: osdmap e363: 8 total, 8 up, 8 in 2024-09-16T08:52:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:52:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-55"} : dispatch 2024-09-16T08:52:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:35 smithi160 ceph-mon[30137]: osdmap e364: 8 total, 8 up, 8 in 2024-09-16T08:52:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:35.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:35 smithi160 ceph-mon[40028]: osdmap e363: 8 total, 8 up, 8 in 2024-09-16T08:52:35.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:52:35.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:35.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-55"} : dispatch 2024-09-16T08:52:35.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:35 smithi160 ceph-mon[40028]: osdmap e364: 8 total, 8 up, 8 in 2024-09-16T08:52:36.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:36 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:36 smithi169 ceph-mon[33465]: pgmap v498: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:52:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:36 smithi169 ceph-mon[33465]: osdmap e365: 8 total, 8 up, 8 in 2024-09-16T08:52:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:36.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:36 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:36 smithi160 ceph-mon[30137]: pgmap v498: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:52:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:36 smithi160 ceph-mon[30137]: osdmap e365: 8 total, 8 up, 8 in 2024-09-16T08:52:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:36 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:36 smithi160 ceph-mon[40028]: pgmap v498: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:52:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:36 smithi160 ceph-mon[40028]: osdmap e365: 8 total, 8 up, 8 in 2024-09-16T08:52:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:38.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:38 smithi169 ceph-mon[33465]: pgmap v501: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:52:38.615 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:38.615 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:38 smithi169 ceph-mon[33465]: osdmap e366: 8 total, 8 up, 8 in 2024-09-16T08:52:38.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:38 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:38.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:38.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-57"} : dispatch 2024-09-16T08:52:38.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:38 smithi160 ceph-mon[40028]: pgmap v501: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:52:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:38 smithi160 ceph-mon[40028]: osdmap e366: 8 total, 8 up, 8 in 2024-09-16T08:52:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:38 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-57"} : dispatch 2024-09-16T08:52:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:38 smithi160 ceph-mon[30137]: pgmap v501: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:52:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:38 smithi160 ceph-mon[30137]: osdmap e366: 8 total, 8 up, 8 in 2024-09-16T08:52:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:38 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:38.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:38.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-57"} : dispatch 2024-09-16T08:52:39.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:39 smithi169 ceph-mon[33465]: osdmap e367: 8 total, 8 up, 8 in 2024-09-16T08:52:39.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:39 smithi169 ceph-mon[33465]: osdmap e368: 8 total, 8 up, 8 in 2024-09-16T08:52:39.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:39.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:39 smithi160 ceph-mon[40028]: osdmap e367: 8 total, 8 up, 8 in 2024-09-16T08:52:39.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:39 smithi160 ceph-mon[40028]: osdmap e368: 8 total, 8 up, 8 in 2024-09-16T08:52:39.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:39.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:39 smithi160 ceph-mon[30137]: osdmap e367: 8 total, 8 up, 8 in 2024-09-16T08:52:39.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:39 smithi160 ceph-mon[30137]: osdmap e368: 8 total, 8 up, 8 in 2024-09-16T08:52:39.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:40.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:40 smithi169 ceph-mon[33465]: pgmap v504: 196 pgs: 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-16T08:52:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:40 smithi169 ceph-mon[33465]: osdmap e369: 8 total, 8 up, 8 in 2024-09-16T08:52:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:52:40.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:40 smithi160 ceph-mon[40028]: pgmap v504: 196 pgs: 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-16T08:52:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:40 smithi160 ceph-mon[40028]: osdmap e369: 8 total, 8 up, 8 in 2024-09-16T08:52:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:52:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:40 smithi160 ceph-mon[30137]: pgmap v504: 196 pgs: 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-16T08:52:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:40 smithi160 ceph-mon[30137]: osdmap e369: 8 total, 8 up, 8 in 2024-09-16T08:52:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:52:41.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:41.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-59"} : dispatch 2024-09-16T08:52:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:41 smithi169 ceph-mon[33465]: osdmap e370: 8 total, 8 up, 8 in 2024-09-16T08:52:41.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:41.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-59"} : dispatch 2024-09-16T08:52:41.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:41 smithi160 ceph-mon[30137]: osdmap e370: 8 total, 8 up, 8 in 2024-09-16T08:52:41.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:41.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-59"} : dispatch 2024-09-16T08:52:41.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:41 smithi160 ceph-mon[40028]: osdmap e370: 8 total, 8 up, 8 in 2024-09-16T08:52:42.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:42 smithi169 ceph-mon[33465]: pgmap v507: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-16T08:52:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:42 smithi169 ceph-mon[33465]: osdmap e371: 8 total, 8 up, 8 in 2024-09-16T08:52:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:42 smithi160 ceph-mon[30137]: pgmap v507: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-16T08:52:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:42 smithi160 ceph-mon[30137]: osdmap e371: 8 total, 8 up, 8 in 2024-09-16T08:52:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:42 smithi160 ceph-mon[40028]: pgmap v507: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-16T08:52:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:42 smithi160 ceph-mon[40028]: osdmap e371: 8 total, 8 up, 8 in 2024-09-16T08:52:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:44.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:44 smithi169 ceph-mon[33465]: pgmap v510: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:52:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:44 smithi169 ceph-mon[33465]: osdmap e372: 8 total, 8 up, 8 in 2024-09-16T08:52:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:52:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:52:44.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:44 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:44 smithi160 ceph-mon[30137]: pgmap v510: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:52:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:44 smithi160 ceph-mon[30137]: osdmap e372: 8 total, 8 up, 8 in 2024-09-16T08:52:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:52:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:52:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:44 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:44.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:44 smithi160 ceph-mon[40028]: pgmap v510: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:52:44.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:44.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:44 smithi160 ceph-mon[40028]: osdmap e372: 8 total, 8 up, 8 in 2024-09-16T08:52:44.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:52:44.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:52:44.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:44 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:45.321 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:52:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:52:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:52:45.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:52:45 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:52:45.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:52:45.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:45 smithi169 ceph-mon[33465]: osdmap e373: 8 total, 8 up, 8 in 2024-09-16T08:52:45.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:45.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-61"} : dispatch 2024-09-16T08:52:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:52:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:45 smithi160 ceph-mon[30137]: osdmap e373: 8 total, 8 up, 8 in 2024-09-16T08:52:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-61"} : dispatch 2024-09-16T08:52:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:52:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:45 smithi160 ceph-mon[40028]: osdmap e373: 8 total, 8 up, 8 in 2024-09-16T08:52:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:52:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-61"} : dispatch 2024-09-16T08:52:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:46 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:46 smithi169 ceph-mon[33465]: pgmap v513: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T08:52:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:46 smithi169 ceph-mon[33465]: osdmap e374: 8 total, 8 up, 8 in 2024-09-16T08:52:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:46 smithi169 ceph-mon[33465]: osdmap e375: 8 total, 8 up, 8 in 2024-09-16T08:52:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:46.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:46 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:46 smithi160 ceph-mon[30137]: pgmap v513: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T08:52:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:46 smithi160 ceph-mon[30137]: osdmap e374: 8 total, 8 up, 8 in 2024-09-16T08:52:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:46 smithi160 ceph-mon[30137]: osdmap e375: 8 total, 8 up, 8 in 2024-09-16T08:52:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:46 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:46 smithi160 ceph-mon[40028]: pgmap v513: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T08:52:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:46 smithi160 ceph-mon[40028]: osdmap e374: 8 total, 8 up, 8 in 2024-09-16T08:52:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:46 smithi160 ceph-mon[40028]: osdmap e375: 8 total, 8 up, 8 in 2024-09-16T08:52:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:52:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:48 smithi169 ceph-mon[33465]: pgmap v516: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T08:52:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:48 smithi169 ceph-mon[33465]: osdmap e376: 8 total, 8 up, 8 in 2024-09-16T08:52:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:52:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:52:48.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:48 smithi160 ceph-mon[40028]: pgmap v516: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T08:52:48.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:48.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:48 smithi160 ceph-mon[40028]: osdmap e376: 8 total, 8 up, 8 in 2024-09-16T08:52:48.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:52:48.701 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:52:48.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:48 smithi160 ceph-mon[30137]: pgmap v516: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T08:52:48.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:48.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:48 smithi160 ceph-mon[30137]: osdmap e376: 8 total, 8 up, 8 in 2024-09-16T08:52:48.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:52:48.702 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:52:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:52:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:49 smithi169 ceph-mon[33465]: osdmap e377: 8 total, 8 up, 8 in 2024-09-16T08:52:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:52:49.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:52:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:49 smithi160 ceph-mon[40028]: osdmap e377: 8 total, 8 up, 8 in 2024-09-16T08:52:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:52:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:52:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:49 smithi160 ceph-mon[30137]: osdmap e377: 8 total, 8 up, 8 in 2024-09-16T08:52:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:52:50.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:50 smithi169 ceph-mon[33465]: pgmap v519: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-16T08:52:50.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:50 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:50.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:50 smithi169 ceph-mon[33465]: osdmap e378: 8 total, 8 up, 8 in 2024-09-16T08:52:50.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:50 smithi160 ceph-mon[40028]: pgmap v519: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-16T08:52:50.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:50 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:50.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:50 smithi160 ceph-mon[40028]: osdmap e378: 8 total, 8 up, 8 in 2024-09-16T08:52:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:50 smithi160 ceph-mon[30137]: pgmap v519: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-16T08:52:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:50 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:52:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:50 smithi160 ceph-mon[30137]: osdmap e378: 8 total, 8 up, 8 in 2024-09-16T08:52:51.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:51 smithi169 ceph-mon[33465]: osdmap e379: 8 total, 8 up, 8 in 2024-09-16T08:52:51.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:51 smithi160 ceph-mon[30137]: osdmap e379: 8 total, 8 up, 8 in 2024-09-16T08:52:51.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:51 smithi160 ceph-mon[40028]: osdmap e379: 8 total, 8 up, 8 in 2024-09-16T08:52:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:52 smithi169 ceph-mon[33465]: pgmap v522: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-16T08:52:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:52 smithi169 ceph-mon[33465]: osdmap e380: 8 total, 8 up, 8 in 2024-09-16T08:52:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:52 smithi160 ceph-mon[30137]: pgmap v522: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-16T08:52:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:52 smithi160 ceph-mon[30137]: osdmap e380: 8 total, 8 up, 8 in 2024-09-16T08:52:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:52 smithi160 ceph-mon[40028]: pgmap v522: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-16T08:52:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:52 smithi160 ceph-mon[40028]: osdmap e380: 8 total, 8 up, 8 in 2024-09-16T08:52:53.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:53 smithi169 ceph-mon[33465]: osdmap e381: 8 total, 8 up, 8 in 2024-09-16T08:52:53.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:53 smithi160 ceph-mon[30137]: osdmap e381: 8 total, 8 up, 8 in 2024-09-16T08:52:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:53 smithi160 ceph-mon[40028]: osdmap e381: 8 total, 8 up, 8 in 2024-09-16T08:52:54.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:54 smithi169 ceph-mon[33465]: pgmap v525: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:52:54.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:54 smithi160 ceph-mon[30137]: pgmap v525: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:52:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:54 smithi160 ceph-mon[40028]: pgmap v525: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:52:55.349 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:52:55 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:52:55.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:52:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:52:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:52:56.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:56 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:56.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:56 smithi160 ceph-mon[30137]: pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-16T08:52:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:56 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:56 smithi160 ceph-mon[40028]: pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-16T08:52:56.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:56 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:52:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:56 smithi169 ceph-mon[33465]: pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-16T08:52:58.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:52:58 smithi160 ceph-mon[30137]: pgmap v527: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 898 B/s wr, 3 op/s 2024-09-16T08:52:58.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:52:58 smithi160 ceph-mon[40028]: pgmap v527: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 898 B/s wr, 3 op/s 2024-09-16T08:52:58.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:52:58 smithi169 ceph-mon[33465]: pgmap v527: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 898 B/s wr, 3 op/s 2024-09-16T08:53:00.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:00 smithi160 ceph-mon[30137]: pgmap v528: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-16T08:53:00.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:00 smithi160 ceph-mon[30137]: osdmap e382: 8 total, 8 up, 8 in 2024-09-16T08:53:00.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:00 smithi160 ceph-mon[40028]: pgmap v528: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-16T08:53:00.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:00 smithi160 ceph-mon[40028]: osdmap e382: 8 total, 8 up, 8 in 2024-09-16T08:53:00.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:00 smithi169 ceph-mon[33465]: pgmap v528: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-16T08:53:00.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:00 smithi169 ceph-mon[33465]: osdmap e382: 8 total, 8 up, 8 in 2024-09-16T08:53:02.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:02 smithi160 ceph-mon[30137]: pgmap v530: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 696 B/s wr, 3 op/s 2024-09-16T08:53:02.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:02 smithi160 ceph-mon[40028]: pgmap v530: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 696 B/s wr, 3 op/s 2024-09-16T08:53:02.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:02 smithi169 ceph-mon[33465]: pgmap v530: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 696 B/s wr, 3 op/s 2024-09-16T08:53:04.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:04 smithi160 ceph-mon[30137]: pgmap v531: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 614 B/s wr, 2 op/s 2024-09-16T08:53:04.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:04 smithi160 ceph-mon[30137]: osdmap e383: 8 total, 8 up, 8 in 2024-09-16T08:53:04.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:53:04.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:04 smithi160 ceph-mon[40028]: pgmap v531: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 614 B/s wr, 2 op/s 2024-09-16T08:53:04.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:04 smithi160 ceph-mon[40028]: osdmap e383: 8 total, 8 up, 8 in 2024-09-16T08:53:04.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:53:04.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:04 smithi169 ceph-mon[33465]: pgmap v531: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 614 B/s wr, 2 op/s 2024-09-16T08:53:04.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:04 smithi169 ceph-mon[33465]: osdmap e383: 8 total, 8 up, 8 in 2024-09-16T08:53:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:53:05.350 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:53:05 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:53:05.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:53:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:53:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:53:06.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:06.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:06 smithi160 ceph-mon[30137]: pgmap v533: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:53:06.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:06.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:06 smithi160 ceph-mon[40028]: pgmap v533: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:53:06.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:06.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:06 smithi169 ceph-mon[33465]: pgmap v533: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:53:08.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:08 smithi169 ceph-mon[33465]: pgmap v534: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:53:08.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:08 smithi160 ceph-mon[30137]: pgmap v534: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:53:08.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:08 smithi160 ceph-mon[40028]: pgmap v534: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:53:10.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:10 smithi160 ceph-mon[30137]: pgmap v535: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:53:10.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:10 smithi160 ceph-mon[30137]: osdmap e384: 8 total, 8 up, 8 in 2024-09-16T08:53:10.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:10 smithi160 ceph-mon[40028]: pgmap v535: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:53:10.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:10 smithi160 ceph-mon[40028]: osdmap e384: 8 total, 8 up, 8 in 2024-09-16T08:53:10.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:10 smithi169 ceph-mon[33465]: pgmap v535: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:53:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:10 smithi169 ceph-mon[33465]: osdmap e384: 8 total, 8 up, 8 in 2024-09-16T08:53:12.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:12 smithi160 ceph-mon[30137]: pgmap v537: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:53:12.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:12 smithi160 ceph-mon[40028]: pgmap v537: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:53:12.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:12 smithi169 ceph-mon[33465]: pgmap v537: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:53:14.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:14 smithi160 ceph-mon[30137]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:53:14.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:14 smithi160 ceph-mon[40028]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:53:14.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:14 smithi169 ceph-mon[33465]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:53:15.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:15 smithi169 ceph-mon[33465]: osdmap e385: 8 total, 8 up, 8 in 2024-09-16T08:53:15.350 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:53:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:53:15.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:15 smithi160 ceph-mon[30137]: osdmap e385: 8 total, 8 up, 8 in 2024-09-16T08:53:15.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:15 smithi160 ceph-mon[40028]: osdmap e385: 8 total, 8 up, 8 in 2024-09-16T08:53:15.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:53:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:53:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:53:16.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:16 smithi169 ceph-mon[33465]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-16T08:53:16.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:16 smithi160 ceph-mon[30137]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-16T08:53:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:16 smithi160 ceph-mon[40028]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-16T08:53:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:18 smithi169 ceph-mon[33465]: pgmap v541: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-16T08:53:18.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:18 smithi160 ceph-mon[30137]: pgmap v541: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-16T08:53:18.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:18 smithi160 ceph-mon[40028]: pgmap v541: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-16T08:53:19.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:53:19.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:53:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:53:20.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:20 smithi169 ceph-mon[33465]: pgmap v542: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:53:20.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:20 smithi160 ceph-mon[30137]: pgmap v542: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:53:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:20 smithi160 ceph-mon[40028]: pgmap v542: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:53:22.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:22 smithi169 ceph-mon[33465]: pgmap v543: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:53:22.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:22 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:53:22.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:22 smithi160 ceph-mon[30137]: pgmap v543: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:53:22.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:22 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:53:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:22 smithi160 ceph-mon[40028]: pgmap v543: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:53:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:22 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:53:23.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:23 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:53:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:23 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:53:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:23 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:53:23.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:23 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:53:23.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:23 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:53:23.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:23 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:53:23.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:23 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:53:23.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:23 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:53:23.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:23 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:53:24.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:24 smithi169 ceph-mon[33465]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:53:24.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:24 smithi169 ceph-mon[33465]: osdmap e386: 8 total, 8 up, 8 in 2024-09-16T08:53:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:53:24.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-63"} : dispatch 2024-09-16T08:53:24.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:24 smithi160 ceph-mon[30137]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:53:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:24 smithi160 ceph-mon[30137]: osdmap e386: 8 total, 8 up, 8 in 2024-09-16T08:53:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:53:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-63"} : dispatch 2024-09-16T08:53:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:24 smithi160 ceph-mon[40028]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:53:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:24 smithi160 ceph-mon[40028]: osdmap e386: 8 total, 8 up, 8 in 2024-09-16T08:53:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:53:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-63"} : dispatch 2024-09-16T08:53:25.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:53:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:53:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:53:25.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:53:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:53:26.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:26.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:26 smithi160 ceph-mon[30137]: pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:53:26.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:26 smithi160 ceph-mon[30137]: osdmap e387: 8 total, 8 up, 8 in 2024-09-16T08:53:26.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:26.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:26 smithi160 ceph-mon[40028]: pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:53:26.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:26 smithi160 ceph-mon[40028]: osdmap e387: 8 total, 8 up, 8 in 2024-09-16T08:53:26.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:26 smithi169 ceph-mon[33465]: pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:53:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:26 smithi169 ceph-mon[33465]: osdmap e387: 8 total, 8 up, 8 in 2024-09-16T08:53:27.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:27 smithi169 ceph-mon[33465]: osdmap e388: 8 total, 8 up, 8 in 2024-09-16T08:53:27.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:53:27.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:27 smithi160 ceph-mon[40028]: osdmap e388: 8 total, 8 up, 8 in 2024-09-16T08:53:27.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:53:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:27 smithi160 ceph-mon[30137]: osdmap e388: 8 total, 8 up, 8 in 2024-09-16T08:53:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:53:28.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:28 smithi169 ceph-mon[33465]: pgmap v549: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:53:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:28 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:53:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:53:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:28 smithi169 ceph-mon[33465]: osdmap e389: 8 total, 8 up, 8 in 2024-09-16T08:53:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:53:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:53:28.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:28 smithi160 ceph-mon[40028]: pgmap v549: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:53:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:28 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:53:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:53:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:28 smithi160 ceph-mon[40028]: osdmap e389: 8 total, 8 up, 8 in 2024-09-16T08:53:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:53:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:53:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:28 smithi160 ceph-mon[30137]: pgmap v549: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:53:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:28 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:53:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:53:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:28 smithi160 ceph-mon[30137]: osdmap e389: 8 total, 8 up, 8 in 2024-09-16T08:53:28.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:53:28.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:53:29.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:53:29.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:29 smithi169 ceph-mon[33465]: osdmap e390: 8 total, 8 up, 8 in 2024-09-16T08:53:29.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:53:29.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:29 smithi160 ceph-mon[40028]: osdmap e390: 8 total, 8 up, 8 in 2024-09-16T08:53:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:53:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:29 smithi160 ceph-mon[30137]: osdmap e390: 8 total, 8 up, 8 in 2024-09-16T08:53:30.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:30 smithi160 ceph-mon[30137]: pgmap v552: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:53:30.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:30 smithi160 ceph-mon[30137]: osdmap e391: 8 total, 8 up, 8 in 2024-09-16T08:53:30.599 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:30 smithi160 ceph-mon[40028]: pgmap v552: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:53:30.599 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:30 smithi160 ceph-mon[40028]: osdmap e391: 8 total, 8 up, 8 in 2024-09-16T08:53:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:30 smithi169 ceph-mon[33465]: pgmap v552: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:53:30.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:30 smithi169 ceph-mon[33465]: osdmap e391: 8 total, 8 up, 8 in 2024-09-16T08:53:31.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:31 smithi169 ceph-mon[33465]: osdmap e392: 8 total, 8 up, 8 in 2024-09-16T08:53:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:31 smithi160 ceph-mon[30137]: osdmap e392: 8 total, 8 up, 8 in 2024-09-16T08:53:31.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:31 smithi160 ceph-mon[40028]: osdmap e392: 8 total, 8 up, 8 in 2024-09-16T08:53:32.501 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:32 smithi169 ceph-mon[33465]: pgmap v555: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-16T08:53:32.501 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:32 smithi169 ceph-mon[33465]: osdmap e393: 8 total, 8 up, 8 in 2024-09-16T08:53:32.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:32 smithi160 ceph-mon[30137]: pgmap v555: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-16T08:53:32.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:32 smithi160 ceph-mon[30137]: osdmap e393: 8 total, 8 up, 8 in 2024-09-16T08:53:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:32 smithi160 ceph-mon[40028]: pgmap v555: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-16T08:53:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:32 smithi160 ceph-mon[40028]: osdmap e393: 8 total, 8 up, 8 in 2024-09-16T08:53:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:34 smithi169 ceph-mon[33465]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-16T08:53:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:53:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:34 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:53:34.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:34 smithi160 ceph-mon[30137]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-16T08:53:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:53:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:34 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:53:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:34 smithi160 ceph-mon[40028]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-16T08:53:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:53:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:34 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:53:35.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:53:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:53:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:53:35.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:53:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:53:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:36 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:36.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:36 smithi169 ceph-mon[33465]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-16T08:53:36.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:36 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:36 smithi160 ceph-mon[30137]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-16T08:53:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:36 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:36 smithi160 ceph-mon[40028]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-16T08:53:38.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:38 smithi169 ceph-mon[33465]: pgmap v559: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 514 B/s wr, 3 op/s 2024-09-16T08:53:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:38 smithi160 ceph-mon[30137]: pgmap v559: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 514 B/s wr, 3 op/s 2024-09-16T08:53:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:38 smithi160 ceph-mon[40028]: pgmap v559: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 514 B/s wr, 3 op/s 2024-09-16T08:53:40.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:40 smithi160 ceph-mon[30137]: pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 457 B/s wr, 3 op/s 2024-09-16T08:53:40.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:40 smithi160 ceph-mon[30137]: osdmap e394: 8 total, 8 up, 8 in 2024-09-16T08:53:40.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:40 smithi160 ceph-mon[40028]: pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 457 B/s wr, 3 op/s 2024-09-16T08:53:40.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:40 smithi160 ceph-mon[40028]: osdmap e394: 8 total, 8 up, 8 in 2024-09-16T08:53:40.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:40 smithi169 ceph-mon[33465]: pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 457 B/s wr, 3 op/s 2024-09-16T08:53:41.280 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:40 smithi169 ceph-mon[33465]: osdmap e394: 8 total, 8 up, 8 in 2024-09-16T08:53:42.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:42 smithi169 ceph-mon[33465]: pgmap v562: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 412 B/s wr, 3 op/s 2024-09-16T08:53:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:53:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-65"} : dispatch 2024-09-16T08:53:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:42 smithi160 ceph-mon[30137]: pgmap v562: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 412 B/s wr, 3 op/s 2024-09-16T08:53:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:53:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-65"} : dispatch 2024-09-16T08:53:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:42 smithi160 ceph-mon[40028]: pgmap v562: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 412 B/s wr, 3 op/s 2024-09-16T08:53:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:53:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-65"} : dispatch 2024-09-16T08:53:43.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:43 smithi169 ceph-mon[33465]: osdmap e395: 8 total, 8 up, 8 in 2024-09-16T08:53:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:43 smithi169 ceph-mon[33465]: osdmap e396: 8 total, 8 up, 8 in 2024-09-16T08:53:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:53:43.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:43 smithi160 ceph-mon[30137]: osdmap e395: 8 total, 8 up, 8 in 2024-09-16T08:53:43.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:43 smithi160 ceph-mon[30137]: osdmap e396: 8 total, 8 up, 8 in 2024-09-16T08:53:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:53:43.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:43 smithi160 ceph-mon[40028]: osdmap e395: 8 total, 8 up, 8 in 2024-09-16T08:53:43.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:43 smithi160 ceph-mon[40028]: osdmap e396: 8 total, 8 up, 8 in 2024-09-16T08:53:43.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:53:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:44 smithi169 ceph-mon[33465]: pgmap v564: 196 pgs: 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:53:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:53:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:44 smithi169 ceph-mon[33465]: osdmap e397: 8 total, 8 up, 8 in 2024-09-16T08:53:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:53:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:44 smithi160 ceph-mon[30137]: pgmap v564: 196 pgs: 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:53:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:53:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:44 smithi160 ceph-mon[30137]: osdmap e397: 8 total, 8 up, 8 in 2024-09-16T08:53:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:53:44.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:44 smithi160 ceph-mon[40028]: pgmap v564: 196 pgs: 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:53:44.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:53:44.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:44 smithi160 ceph-mon[40028]: osdmap e397: 8 total, 8 up, 8 in 2024-09-16T08:53:44.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:53:45.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:53:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:53:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:53:45.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:53:45 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:53:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:46 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:46 smithi169 ceph-mon[33465]: pgmap v567: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:53:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:46 smithi169 ceph-mon[33465]: osdmap e398: 8 total, 8 up, 8 in 2024-09-16T08:53:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:53:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-67"} : dispatch 2024-09-16T08:53:46.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:46 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:46 smithi160 ceph-mon[30137]: pgmap v567: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:53:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:46 smithi160 ceph-mon[30137]: osdmap e398: 8 total, 8 up, 8 in 2024-09-16T08:53:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:53:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-67"} : dispatch 2024-09-16T08:53:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:46 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:46 smithi160 ceph-mon[40028]: pgmap v567: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:53:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:46 smithi160 ceph-mon[40028]: osdmap e398: 8 total, 8 up, 8 in 2024-09-16T08:53:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:53:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-67"} : dispatch 2024-09-16T08:53:47.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:47 smithi169 ceph-mon[33465]: osdmap e399: 8 total, 8 up, 8 in 2024-09-16T08:53:47.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:47 smithi160 ceph-mon[30137]: osdmap e399: 8 total, 8 up, 8 in 2024-09-16T08:53:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:47 smithi160 ceph-mon[40028]: osdmap e399: 8 total, 8 up, 8 in 2024-09-16T08:53:48.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:48 smithi169 ceph-mon[33465]: pgmap v570: 196 pgs: 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:53:48.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:48 smithi169 ceph-mon[33465]: osdmap e400: 8 total, 8 up, 8 in 2024-09-16T08:53:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:53:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:48 smithi160 ceph-mon[30137]: pgmap v570: 196 pgs: 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:53:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:48 smithi160 ceph-mon[30137]: osdmap e400: 8 total, 8 up, 8 in 2024-09-16T08:53:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:53:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:48 smithi160 ceph-mon[40028]: pgmap v570: 196 pgs: 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:53:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:48 smithi160 ceph-mon[40028]: osdmap e400: 8 total, 8 up, 8 in 2024-09-16T08:53:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:53:49.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:53:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:49 smithi169 ceph-mon[33465]: osdmap e401: 8 total, 8 up, 8 in 2024-09-16T08:53:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:53:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:53:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-69"} : dispatch 2024-09-16T08:53:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:53:49.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:53:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:49 smithi160 ceph-mon[30137]: osdmap e401: 8 total, 8 up, 8 in 2024-09-16T08:53:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:53:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:53:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-69"} : dispatch 2024-09-16T08:53:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:53:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:53:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:49 smithi160 ceph-mon[40028]: osdmap e401: 8 total, 8 up, 8 in 2024-09-16T08:53:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:53:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:53:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-69"} : dispatch 2024-09-16T08:53:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:53:50.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:50 smithi169 ceph-mon[33465]: pgmap v573: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:53:50.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:50 smithi169 ceph-mon[33465]: osdmap e402: 8 total, 8 up, 8 in 2024-09-16T08:53:50.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:50 smithi160 ceph-mon[30137]: pgmap v573: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:53:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:50 smithi160 ceph-mon[30137]: osdmap e402: 8 total, 8 up, 8 in 2024-09-16T08:53:50.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:50 smithi160 ceph-mon[40028]: pgmap v573: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:53:50.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:50 smithi160 ceph-mon[40028]: osdmap e402: 8 total, 8 up, 8 in 2024-09-16T08:53:51.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:51 smithi169 ceph-mon[33465]: osdmap e403: 8 total, 8 up, 8 in 2024-09-16T08:53:51.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:53:51.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:51 smithi160 ceph-mon[30137]: osdmap e403: 8 total, 8 up, 8 in 2024-09-16T08:53:51.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:53:51.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:51 smithi160 ceph-mon[40028]: osdmap e403: 8 total, 8 up, 8 in 2024-09-16T08:53:51.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:53:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:52 smithi169 ceph-mon[33465]: pgmap v576: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:53:52.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:52 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:53:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:53:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:52 smithi169 ceph-mon[33465]: osdmap e404: 8 total, 8 up, 8 in 2024-09-16T08:53:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:53:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:53:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:52 smithi160 ceph-mon[30137]: pgmap v576: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:53:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:52 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:53:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:53:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:52 smithi160 ceph-mon[30137]: osdmap e404: 8 total, 8 up, 8 in 2024-09-16T08:53:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:53:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:53:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:52 smithi160 ceph-mon[40028]: pgmap v576: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:53:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:52 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:53:52.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:53:52.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:52 smithi160 ceph-mon[40028]: osdmap e404: 8 total, 8 up, 8 in 2024-09-16T08:53:52.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:53:52.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:53:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:53:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:53 smithi169 ceph-mon[33465]: osdmap e405: 8 total, 8 up, 8 in 2024-09-16T08:53:53.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:53:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:53 smithi160 ceph-mon[30137]: osdmap e405: 8 total, 8 up, 8 in 2024-09-16T08:53:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:53:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:53 smithi160 ceph-mon[40028]: osdmap e405: 8 total, 8 up, 8 in 2024-09-16T08:53:54.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:54 smithi169 ceph-mon[33465]: pgmap v579: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:53:54.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:54 smithi169 ceph-mon[33465]: osdmap e406: 8 total, 8 up, 8 in 2024-09-16T08:53:54.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:54 smithi160 ceph-mon[30137]: pgmap v579: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:53:54.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:54 smithi160 ceph-mon[30137]: osdmap e406: 8 total, 8 up, 8 in 2024-09-16T08:53:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:54 smithi160 ceph-mon[40028]: pgmap v579: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:53:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:54 smithi160 ceph-mon[40028]: osdmap e406: 8 total, 8 up, 8 in 2024-09-16T08:53:55.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:53:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:53:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:53:55.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:53:55 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:53:56.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:56 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:56.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:56 smithi169 ceph-mon[33465]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-09-16T08:53:56.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:56 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:56 smithi160 ceph-mon[30137]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-09-16T08:53:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:56 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:53:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:56 smithi160 ceph-mon[40028]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-09-16T08:53:58.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:58 smithi169 ceph-mon[33465]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-16T08:53:58.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:58 smithi160 ceph-mon[30137]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-16T08:53:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:58 smithi160 ceph-mon[40028]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-16T08:53:59.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:53:59 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:53:59.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:53:59 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:53:59.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:53:59 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:54:00.599 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:00 smithi160 ceph-mon[30137]: pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-09-16T08:54:00.599 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:00 smithi160 ceph-mon[40028]: pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-09-16T08:54:00.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:00 smithi169 ceph-mon[33465]: pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-09-16T08:54:02.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:02 smithi169 ceph-mon[33465]: pgmap v584: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:54:02.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:02 smithi160 ceph-mon[30137]: pgmap v584: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:54:02.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:02 smithi160 ceph-mon[40028]: pgmap v584: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:54:04.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:04 smithi169 ceph-mon[33465]: pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-16T08:54:04.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1e", "id": [6, 2]} : dispatch 2024-09-16T08:54:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1f", "id": [6, 3]} : dispatch 2024-09-16T08:54:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:54:04.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:04 smithi160 ceph-mon[30137]: pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-16T08:54:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1e", "id": [6, 2]} : dispatch 2024-09-16T08:54:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1f", "id": [6, 3]} : dispatch 2024-09-16T08:54:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:54:04.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:04 smithi160 ceph-mon[40028]: pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-16T08:54:04.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1e", "id": [6, 2]} : dispatch 2024-09-16T08:54:04.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1f", "id": [6, 3]} : dispatch 2024-09-16T08:54:04.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:54:05.305 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:54:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:54:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:54:05.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:54:05 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:54:05.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:05 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1e", "id": [6, 2]}]': finished 2024-09-16T08:54:05.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:05 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1f", "id": [6, 3]}]': finished 2024-09-16T08:54:05.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:05 smithi169 ceph-mon[33465]: osdmap e407: 8 total, 8 up, 8 in 2024-09-16T08:54:05.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:05 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1e", "id": [6, 2]}]': finished 2024-09-16T08:54:05.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:05 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1f", "id": [6, 3]}]': finished 2024-09-16T08:54:05.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:05 smithi160 ceph-mon[30137]: osdmap e407: 8 total, 8 up, 8 in 2024-09-16T08:54:05.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:05 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1e", "id": [6, 2]}]': finished 2024-09-16T08:54:05.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:05 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1f", "id": [6, 3]}]': finished 2024-09-16T08:54:05.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:05 smithi160 ceph-mon[40028]: osdmap e407: 8 total, 8 up, 8 in 2024-09-16T08:54:06.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:06 smithi169 ceph-mon[33465]: pgmap v587: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-16T08:54:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:06 smithi169 ceph-mon[33465]: osdmap e408: 8 total, 8 up, 8 in 2024-09-16T08:54:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:06 smithi160 ceph-mon[30137]: pgmap v587: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-16T08:54:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:06 smithi160 ceph-mon[30137]: osdmap e408: 8 total, 8 up, 8 in 2024-09-16T08:54:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:06 smithi160 ceph-mon[40028]: pgmap v587: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-16T08:54:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:06 smithi160 ceph-mon[40028]: osdmap e408: 8 total, 8 up, 8 in 2024-09-16T08:54:08.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:08 smithi169 ceph-mon[33465]: pgmap v589: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 383 B/s wr, 2 op/s 2024-09-16T08:54:08.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:08 smithi160 ceph-mon[30137]: pgmap v589: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 383 B/s wr, 2 op/s 2024-09-16T08:54:08.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:08 smithi160 ceph-mon[40028]: pgmap v589: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 383 B/s wr, 2 op/s 2024-09-16T08:54:10.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:10 smithi169 ceph-mon[33465]: pgmap v590: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 383 B/s wr, 3 op/s 2024-09-16T08:54:10.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:10 smithi160 ceph-mon[30137]: pgmap v590: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 383 B/s wr, 3 op/s 2024-09-16T08:54:10.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:10 smithi160 ceph-mon[40028]: pgmap v590: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 383 B/s wr, 3 op/s 2024-09-16T08:54:12.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:12 smithi169 ceph-mon[33465]: pgmap v591: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 383 B/s wr, 3 op/s 2024-09-16T08:54:12.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:12 smithi160 ceph-mon[30137]: pgmap v591: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 383 B/s wr, 3 op/s 2024-09-16T08:54:12.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:12 smithi160 ceph-mon[40028]: pgmap v591: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 383 B/s wr, 3 op/s 2024-09-16T08:54:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:14 smithi169 ceph-mon[33465]: pgmap v592: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 575 B/s rd, 0 op/s 2024-09-16T08:54:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:14 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:54:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:14 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-71"} : dispatch 2024-09-16T08:54:14.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:14 smithi160 ceph-mon[30137]: pgmap v592: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 575 B/s rd, 0 op/s 2024-09-16T08:54:14.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:54:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-71"} : dispatch 2024-09-16T08:54:14.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:14 smithi160 ceph-mon[40028]: pgmap v592: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 575 B/s rd, 0 op/s 2024-09-16T08:54:14.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:14 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:54:14.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:14 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-71"} : dispatch 2024-09-16T08:54:15.333 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:54:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:54:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:54:15.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:54:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:54:15.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:15 smithi169 ceph-mon[33465]: osdmap e409: 8 total, 8 up, 8 in 2024-09-16T08:54:15.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:15 smithi169 ceph-mon[33465]: osdmap e410: 8 total, 8 up, 8 in 2024-09-16T08:54:15.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:15 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:54:15.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:15 smithi160 ceph-mon[30137]: osdmap e409: 8 total, 8 up, 8 in 2024-09-16T08:54:15.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:15 smithi160 ceph-mon[30137]: osdmap e410: 8 total, 8 up, 8 in 2024-09-16T08:54:15.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:15 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:54:15.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:15 smithi160 ceph-mon[40028]: osdmap e409: 8 total, 8 up, 8 in 2024-09-16T08:54:15.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:15 smithi160 ceph-mon[40028]: osdmap e410: 8 total, 8 up, 8 in 2024-09-16T08:54:15.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:15 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:54:16.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:16 smithi169 ceph-mon[33465]: pgmap v594: 196 pgs: 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:54:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:54:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:16 smithi169 ceph-mon[33465]: osdmap e411: 8 total, 8 up, 8 in 2024-09-16T08:54:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:54:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:16 smithi160 ceph-mon[30137]: pgmap v594: 196 pgs: 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:54:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:54:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:16 smithi160 ceph-mon[30137]: osdmap e411: 8 total, 8 up, 8 in 2024-09-16T08:54:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:54:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:16 smithi160 ceph-mon[40028]: pgmap v594: 196 pgs: 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:54:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:54:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:16 smithi160 ceph-mon[40028]: osdmap e411: 8 total, 8 up, 8 in 2024-09-16T08:54:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:54:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:18 smithi169 ceph-mon[33465]: pgmap v597: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:54:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:18 smithi169 ceph-mon[33465]: osdmap e412: 8 total, 8 up, 8 in 2024-09-16T08:54:18.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:18 smithi160 ceph-mon[40028]: pgmap v597: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:54:18.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:18 smithi160 ceph-mon[40028]: osdmap e412: 8 total, 8 up, 8 in 2024-09-16T08:54:18.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:18 smithi160 ceph-mon[30137]: pgmap v597: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:54:18.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:18 smithi160 ceph-mon[30137]: osdmap e412: 8 total, 8 up, 8 in 2024-09-16T08:54:19.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:19 smithi169 ceph-mon[33465]: osdmap e413: 8 total, 8 up, 8 in 2024-09-16T08:54:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:54:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:19 smithi160 ceph-mon[40028]: osdmap e413: 8 total, 8 up, 8 in 2024-09-16T08:54:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:54:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:19 smithi160 ceph-mon[30137]: osdmap e413: 8 total, 8 up, 8 in 2024-09-16T08:54:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:54:20.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:20 smithi169 ceph-mon[33465]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T08:54:20.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:20 smithi160 ceph-mon[30137]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T08:54:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:20 smithi160 ceph-mon[40028]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T08:54:22.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:22 smithi169 ceph-mon[33465]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-16T08:54:22.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:22 smithi160 ceph-mon[30137]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-16T08:54:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:22 smithi160 ceph-mon[40028]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-16T08:54:23.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:23 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:54:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:23 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:54:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:23 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:54:23.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:23 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:54:23.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:23 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:54:23.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:23 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:54:23.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:23 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:54:23.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:23 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:54:23.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:23 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:54:23.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:23 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:54:23.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:23 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:54:23.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:23 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:54:24.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:24 smithi169 ceph-mon[33465]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-16T08:54:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:24 smithi160 ceph-mon[30137]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-16T08:54:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:24 smithi160 ceph-mon[40028]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-16T08:54:25.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:54:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:54:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:54:25.600 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:54:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:54:26.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:26.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:26 smithi169 ceph-mon[33465]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-09-16T08:54:26.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:26 smithi160 ceph-mon[30137]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-09-16T08:54:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:26 smithi160 ceph-mon[40028]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-09-16T08:54:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:28 smithi160 ceph-mon[30137]: pgmap v604: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 726 B/s wr, 3 op/s 2024-09-16T08:54:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:28 smithi160 ceph-mon[40028]: pgmap v604: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 726 B/s wr, 3 op/s 2024-09-16T08:54:28.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:28 smithi169 ceph-mon[33465]: pgmap v604: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 726 B/s wr, 3 op/s 2024-09-16T08:54:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:54:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-73"} : dispatch 2024-09-16T08:54:29.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:54:29.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-73"} : dispatch 2024-09-16T08:54:29.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:54:29.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-73"} : dispatch 2024-09-16T08:54:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:30 smithi160 ceph-mon[30137]: pgmap v605: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 659 B/s wr, 3 op/s 2024-09-16T08:54:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:30 smithi160 ceph-mon[30137]: osdmap e414: 8 total, 8 up, 8 in 2024-09-16T08:54:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:30 smithi160 ceph-mon[40028]: pgmap v605: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 659 B/s wr, 3 op/s 2024-09-16T08:54:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:30 smithi160 ceph-mon[40028]: osdmap e414: 8 total, 8 up, 8 in 2024-09-16T08:54:30.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:30 smithi169 ceph-mon[33465]: pgmap v605: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 659 B/s wr, 3 op/s 2024-09-16T08:54:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:30 smithi169 ceph-mon[33465]: osdmap e414: 8 total, 8 up, 8 in 2024-09-16T08:54:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:31 smithi160 ceph-mon[30137]: osdmap e415: 8 total, 8 up, 8 in 2024-09-16T08:54:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:54:31.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:31 smithi160 ceph-mon[40028]: osdmap e415: 8 total, 8 up, 8 in 2024-09-16T08:54:31.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:54:31.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:31 smithi169 ceph-mon[33465]: osdmap e415: 8 total, 8 up, 8 in 2024-09-16T08:54:31.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:54:32.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:32 smithi160 ceph-mon[30137]: pgmap v608: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-16T08:54:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:32 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:54:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:54:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:32 smithi160 ceph-mon[30137]: osdmap e416: 8 total, 8 up, 8 in 2024-09-16T08:54:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:54:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:32 smithi160 ceph-mon[40028]: pgmap v608: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-16T08:54:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:32 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:54:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:54:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:32 smithi160 ceph-mon[40028]: osdmap e416: 8 total, 8 up, 8 in 2024-09-16T08:54:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:54:32.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:32 smithi169 ceph-mon[33465]: pgmap v608: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-16T08:54:32.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:32 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:54:32.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:54:32.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:32 smithi169 ceph-mon[33465]: osdmap e416: 8 total, 8 up, 8 in 2024-09-16T08:54:32.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:54:33.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:33 smithi160 ceph-mon[30137]: osdmap e417: 8 total, 8 up, 8 in 2024-09-16T08:54:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:33 smithi160 ceph-mon[40028]: osdmap e417: 8 total, 8 up, 8 in 2024-09-16T08:54:33.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:33 smithi169 ceph-mon[33465]: osdmap e417: 8 total, 8 up, 8 in 2024-09-16T08:54:34.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:34 smithi160 ceph-mon[30137]: pgmap v611: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:54:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:34 smithi160 ceph-mon[30137]: osdmap e418: 8 total, 8 up, 8 in 2024-09-16T08:54:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:54:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-75"} : dispatch 2024-09-16T08:54:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:54:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:34 smithi160 ceph-mon[40028]: pgmap v611: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:54:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:34 smithi160 ceph-mon[40028]: osdmap e418: 8 total, 8 up, 8 in 2024-09-16T08:54:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:54:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-75"} : dispatch 2024-09-16T08:54:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:54:34.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:34 smithi169 ceph-mon[33465]: pgmap v611: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:54:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:34 smithi169 ceph-mon[33465]: osdmap e418: 8 total, 8 up, 8 in 2024-09-16T08:54:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:54:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-75"} : dispatch 2024-09-16T08:54:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:54:35.406 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:54:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:54:35.406 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:54:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:54:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:54:35.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:35 smithi160 ceph-mon[30137]: osdmap e419: 8 total, 8 up, 8 in 2024-09-16T08:54:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:35 smithi160 ceph-mon[40028]: osdmap e419: 8 total, 8 up, 8 in 2024-09-16T08:54:35.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:35 smithi169 ceph-mon[33465]: osdmap e419: 8 total, 8 up, 8 in 2024-09-16T08:54:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:36 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:36 smithi160 ceph-mon[30137]: pgmap v614: 196 pgs: 196 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T08:54:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:36 smithi160 ceph-mon[30137]: osdmap e420: 8 total, 8 up, 8 in 2024-09-16T08:54:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:54:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:36 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:36 smithi160 ceph-mon[40028]: pgmap v614: 196 pgs: 196 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T08:54:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:36 smithi160 ceph-mon[40028]: osdmap e420: 8 total, 8 up, 8 in 2024-09-16T08:54:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:54:36.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:36 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:36 smithi169 ceph-mon[33465]: pgmap v614: 196 pgs: 196 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T08:54:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:36 smithi169 ceph-mon[33465]: osdmap e420: 8 total, 8 up, 8 in 2024-09-16T08:54:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:54:37.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:54:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:37 smithi160 ceph-mon[30137]: osdmap e421: 8 total, 8 up, 8 in 2024-09-16T08:54:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:54:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:54:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:37 smithi160 ceph-mon[40028]: osdmap e421: 8 total, 8 up, 8 in 2024-09-16T08:54:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:54:37.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:54:37.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:37 smithi169 ceph-mon[33465]: osdmap e421: 8 total, 8 up, 8 in 2024-09-16T08:54:37.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:54:38.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:38 smithi160 ceph-mon[40028]: pgmap v617: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T08:54:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:38 smithi160 ceph-mon[40028]: osdmap e422: 8 total, 8 up, 8 in 2024-09-16T08:54:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:54:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-77"} : dispatch 2024-09-16T08:54:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:38 smithi160 ceph-mon[30137]: pgmap v617: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T08:54:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:38 smithi160 ceph-mon[30137]: osdmap e422: 8 total, 8 up, 8 in 2024-09-16T08:54:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:54:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-77"} : dispatch 2024-09-16T08:54:38.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:38 smithi169 ceph-mon[33465]: pgmap v617: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T08:54:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:38 smithi169 ceph-mon[33465]: osdmap e422: 8 total, 8 up, 8 in 2024-09-16T08:54:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:54:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-77"} : dispatch 2024-09-16T08:54:39.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:39 smithi160 ceph-mon[30137]: osdmap e423: 8 total, 8 up, 8 in 2024-09-16T08:54:39.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:39 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:54:39.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:39 smithi160 ceph-mon[30137]: osdmap e424: 8 total, 8 up, 8 in 2024-09-16T08:54:39.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:54:39.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:39 smithi160 ceph-mon[40028]: osdmap e423: 8 total, 8 up, 8 in 2024-09-16T08:54:39.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:39 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:54:39.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:39 smithi160 ceph-mon[40028]: osdmap e424: 8 total, 8 up, 8 in 2024-09-16T08:54:39.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:54:39.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:39 smithi169 ceph-mon[33465]: osdmap e423: 8 total, 8 up, 8 in 2024-09-16T08:54:39.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:39 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:54:39.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:39 smithi169 ceph-mon[33465]: osdmap e424: 8 total, 8 up, 8 in 2024-09-16T08:54:39.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:54:40.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:40 smithi169 ceph-mon[33465]: pgmap v620: 196 pgs: 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T08:54:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:54:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:40 smithi169 ceph-mon[33465]: osdmap e425: 8 total, 8 up, 8 in 2024-09-16T08:54:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:54:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:40 smithi160 ceph-mon[30137]: pgmap v620: 196 pgs: 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T08:54:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:54:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:40 smithi160 ceph-mon[30137]: osdmap e425: 8 total, 8 up, 8 in 2024-09-16T08:54:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:54:40.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:40 smithi160 ceph-mon[40028]: pgmap v620: 196 pgs: 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T08:54:40.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:54:40.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:40 smithi160 ceph-mon[40028]: osdmap e425: 8 total, 8 up, 8 in 2024-09-16T08:54:40.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:54:42.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:42 smithi160 ceph-mon[30137]: pgmap v623: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T08:54:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:42 smithi160 ceph-mon[30137]: osdmap e426: 8 total, 8 up, 8 in 2024-09-16T08:54:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:42 smithi160 ceph-mon[40028]: pgmap v623: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T08:54:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:42 smithi160 ceph-mon[40028]: osdmap e426: 8 total, 8 up, 8 in 2024-09-16T08:54:42.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:42 smithi169 ceph-mon[33465]: pgmap v623: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T08:54:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:42 smithi169 ceph-mon[33465]: osdmap e426: 8 total, 8 up, 8 in 2024-09-16T08:54:43.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:43 smithi169 ceph-mon[33465]: osdmap e427: 8 total, 8 up, 8 in 2024-09-16T08:54:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "304.4"} : dispatch 2024-09-16T08:54:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:43 smithi169 ceph-mon[33465]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.4"}]: dispatch 2024-09-16T08:54:43.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:43 smithi160 ceph-mon[30137]: osdmap e427: 8 total, 8 up, 8 in 2024-09-16T08:54:43.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "304.4"} : dispatch 2024-09-16T08:54:43.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:43 smithi160 ceph-mon[30137]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.4"}]: dispatch 2024-09-16T08:54:43.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:43 smithi160 ceph-mon[40028]: osdmap e427: 8 total, 8 up, 8 in 2024-09-16T08:54:43.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "304.4"} : dispatch 2024-09-16T08:54:43.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:43 smithi160 ceph-mon[40028]: from='mon.0 v1:172.21.15.160:0/2583349001' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.4"}]: dispatch 2024-09-16T08:54:44.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:44 smithi169 ceph-mon[33465]: pgmap v626: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:54:44.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:44 smithi169 ceph-mon[33465]: 304.4 deep-scrub starts 2024-09-16T08:54:44.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:44 smithi169 ceph-mon[33465]: 304.4 deep-scrub ok 2024-09-16T08:54:44.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:44 smithi160 ceph-mon[30137]: pgmap v626: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:54:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:44 smithi160 ceph-mon[30137]: 304.4 deep-scrub starts 2024-09-16T08:54:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:44 smithi160 ceph-mon[30137]: 304.4 deep-scrub ok 2024-09-16T08:54:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:44 smithi160 ceph-mon[40028]: pgmap v626: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:54:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:44 smithi160 ceph-mon[40028]: 304.4 deep-scrub starts 2024-09-16T08:54:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:44 smithi160 ceph-mon[40028]: 304.4 deep-scrub ok 2024-09-16T08:54:45.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:54:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:54:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:54:45.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:54:45 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:54:46.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:46 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:46.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:46 smithi169 ceph-mon[33465]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 887 B/s rd, 887 B/s wr, 2 op/s 2024-09-16T08:54:46.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:46 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:46.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:46 smithi160 ceph-mon[30137]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 887 B/s rd, 887 B/s wr, 2 op/s 2024-09-16T08:54:46.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:46 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:46.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:46 smithi160 ceph-mon[40028]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 887 B/s rd, 887 B/s wr, 2 op/s 2024-09-16T08:54:48.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:48 smithi169 ceph-mon[33465]: pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 756 B/s rd, 756 B/s wr, 2 op/s 2024-09-16T08:54:48.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:48 smithi160 ceph-mon[30137]: pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 756 B/s rd, 756 B/s wr, 2 op/s 2024-09-16T08:54:48.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:48 smithi160 ceph-mon[40028]: pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 756 B/s rd, 756 B/s wr, 2 op/s 2024-09-16T08:54:49.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:54:49.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:54:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:54:50.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:50 smithi169 ceph-mon[33465]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-09-16T08:54:50.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:50 smithi160 ceph-mon[30137]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-09-16T08:54:50.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:50 smithi160 ceph-mon[40028]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-09-16T08:54:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:52 smithi169 ceph-mon[33465]: pgmap v630: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 524 B/s wr, 2 op/s 2024-09-16T08:54:52.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:52 smithi160 ceph-mon[30137]: pgmap v630: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 524 B/s wr, 2 op/s 2024-09-16T08:54:52.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:52 smithi160 ceph-mon[40028]: pgmap v630: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 524 B/s wr, 2 op/s 2024-09-16T08:54:54.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:54 smithi169 ceph-mon[33465]: pgmap v631: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 475 B/s wr, 1 op/s 2024-09-16T08:54:54.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:54 smithi160 ceph-mon[30137]: pgmap v631: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 475 B/s wr, 1 op/s 2024-09-16T08:54:54.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:54 smithi160 ceph-mon[40028]: pgmap v631: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 475 B/s wr, 1 op/s 2024-09-16T08:54:55.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:54:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:54:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:54:55.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:54:55 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:54:56.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:56 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:56.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:56 smithi169 ceph-mon[33465]: pgmap v632: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-16T08:54:56.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:56 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:56 smithi160 ceph-mon[30137]: pgmap v632: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-16T08:54:56.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:56 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:54:56.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:56 smithi160 ceph-mon[40028]: pgmap v632: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-16T08:54:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:54:58 smithi169 ceph-mon[33465]: pgmap v633: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-16T08:54:58.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:54:58 smithi160 ceph-mon[30137]: pgmap v633: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-16T08:54:58.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:54:58 smithi160 ceph-mon[40028]: pgmap v633: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-16T08:55:00.750 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:00 smithi160 ceph-mon[30137]: pgmap v634: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:55:00.750 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:00 smithi160 ceph-mon[40028]: pgmap v634: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:55:00.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:00 smithi169 ceph-mon[33465]: pgmap v634: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:55:02.849 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 08:55:02 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4[38096]: 2024-09-16T08:55:02.467+0000 7fee267b0640 -1 snap_mapper.add_oid found existing snaps mapped on 304:22608cfe:test-rados-api-smithi160-76079-80::foo:2, removing 2024-09-16T08:55:02.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:02 smithi169 ceph-mon[33465]: pgmap v635: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:55:02.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:02 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:02.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:02 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-79"} : dispatch 2024-09-16T08:55:02.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:02 smithi160 ceph-mon[40028]: pgmap v635: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:55:02.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:02 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:02.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:02 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-79"} : dispatch 2024-09-16T08:55:02.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:02 smithi160 ceph-mon[30137]: pgmap v635: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:55:02.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:02 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:02.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:02 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-79"} : dispatch 2024-09-16T08:55:02.949 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 08:55:02 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-3[61676]: 2024-09-16T08:55:02.467+0000 7f72b1cb2640 -1 snap_mapper.add_oid found existing snaps mapped on 304:22608cfe:test-rados-api-smithi160-76079-80::foo:2, removing 2024-09-16T08:55:02.949 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 08:55:02 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-2[55873]: 2024-09-16T08:55:02.467+0000 7f1fbdedb640 -1 snap_mapper.add_oid found existing snaps mapped on 304:22608cfe:test-rados-api-smithi160-76079-80::foo:2, removing 2024-09-16T08:55:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:04 smithi169 ceph-mon[33465]: pgmap v636: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:55:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:04 smithi169 ceph-mon[33465]: osdmap e428: 8 total, 8 up, 8 in 2024-09-16T08:55:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:55:04.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:04 smithi160 ceph-mon[30137]: pgmap v636: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:55:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:04 smithi160 ceph-mon[30137]: osdmap e428: 8 total, 8 up, 8 in 2024-09-16T08:55:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:55:04.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:04 smithi160 ceph-mon[40028]: pgmap v636: 228 pgs: 228 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:55:04.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:04 smithi160 ceph-mon[40028]: osdmap e428: 8 total, 8 up, 8 in 2024-09-16T08:55:04.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:55:05.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:55:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:55:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:55:05.500 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:55:05 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:55:05.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:05 smithi169 ceph-mon[33465]: osdmap e429: 8 total, 8 up, 8 in 2024-09-16T08:55:05.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:05.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:05 smithi160 ceph-mon[30137]: osdmap e429: 8 total, 8 up, 8 in 2024-09-16T08:55:05.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:05.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:05 smithi160 ceph-mon[40028]: osdmap e429: 8 total, 8 up, 8 in 2024-09-16T08:55:05.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:06.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:06 smithi169 ceph-mon[33465]: pgmap v639: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-16T08:55:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:06 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:06 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:06 smithi169 ceph-mon[33465]: osdmap e430: 8 total, 8 up, 8 in 2024-09-16T08:55:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:06 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:06 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:55:06.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:06 smithi160 ceph-mon[30137]: pgmap v639: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-16T08:55:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:06 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:06 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:06 smithi160 ceph-mon[30137]: osdmap e430: 8 total, 8 up, 8 in 2024-09-16T08:55:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:06 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:06 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:55:06.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:06.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:06 smithi160 ceph-mon[40028]: pgmap v639: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-16T08:55:06.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:06 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:06.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:06 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:06.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:06 smithi160 ceph-mon[40028]: osdmap e430: 8 total, 8 up, 8 in 2024-09-16T08:55:06.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:06 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:06.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:06 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:55:07.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:07 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:55:07.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:07 smithi169 ceph-mon[33465]: osdmap e431: 8 total, 8 up, 8 in 2024-09-16T08:55:07.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:07 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:07.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:07 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:55:07.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:07 smithi160 ceph-mon[30137]: osdmap e431: 8 total, 8 up, 8 in 2024-09-16T08:55:07.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:07 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:07.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:07 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:55:07.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:07 smithi160 ceph-mon[40028]: osdmap e431: 8 total, 8 up, 8 in 2024-09-16T08:55:07.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:07 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:08.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:08 smithi169 ceph-mon[33465]: pgmap v642: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-16T08:55:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:55:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:08 smithi169 ceph-mon[33465]: osdmap e432: 8 total, 8 up, 8 in 2024-09-16T08:55:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-16T08:55:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:08 smithi160 ceph-mon[30137]: pgmap v642: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-16T08:55:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:55:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:08 smithi160 ceph-mon[30137]: osdmap e432: 8 total, 8 up, 8 in 2024-09-16T08:55:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-16T08:55:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:08 smithi160 ceph-mon[40028]: pgmap v642: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-16T08:55:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:55:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:08 smithi160 ceph-mon[40028]: osdmap e432: 8 total, 8 up, 8 in 2024-09-16T08:55:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-16T08:55:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-16T08:55:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:09 smithi169 ceph-mon[33465]: osdmap e433: 8 total, 8 up, 8 in 2024-09-16T08:55:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:55:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:55:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:09 smithi169 ceph-mon[33465]: osdmap e434: 8 total, 8 up, 8 in 2024-09-16T08:55:09.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-16T08:55:09.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:09 smithi160 ceph-mon[30137]: osdmap e433: 8 total, 8 up, 8 in 2024-09-16T08:55:09.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:55:09.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:55:09.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:09 smithi160 ceph-mon[30137]: osdmap e434: 8 total, 8 up, 8 in 2024-09-16T08:55:09.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-16T08:55:09.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:09 smithi160 ceph-mon[40028]: osdmap e433: 8 total, 8 up, 8 in 2024-09-16T08:55:09.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:55:09.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:55:09.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:09 smithi160 ceph-mon[40028]: osdmap e434: 8 total, 8 up, 8 in 2024-09-16T08:55:10.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:10 smithi169 ceph-mon[33465]: pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:55:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-16T08:55:10.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:10 smithi160 ceph-mon[30137]: pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:55:10.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-16T08:55:10.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:10 smithi160 ceph-mon[40028]: pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:55:10.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-16T08:55:11.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-16T08:55:11.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:11 smithi169 ceph-mon[33465]: osdmap e435: 8 total, 8 up, 8 in 2024-09-16T08:55:11.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-16T08:55:11.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-16T08:55:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:11 smithi160 ceph-mon[30137]: osdmap e435: 8 total, 8 up, 8 in 2024-09-16T08:55:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-16T08:55:11.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-16T08:55:11.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:11 smithi160 ceph-mon[40028]: osdmap e435: 8 total, 8 up, 8 in 2024-09-16T08:55:11.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-16T08:55:12.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:12 smithi169 ceph-mon[33465]: pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:55:12.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-16T08:55:12.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:12 smithi169 ceph-mon[33465]: osdmap e436: 8 total, 8 up, 8 in 2024-09-16T08:55:12.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:55:12.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:12 smithi160 ceph-mon[30137]: pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:55:12.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-16T08:55:12.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:12 smithi160 ceph-mon[30137]: osdmap e436: 8 total, 8 up, 8 in 2024-09-16T08:55:12.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:55:12.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:12 smithi160 ceph-mon[40028]: pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:55:12.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-16T08:55:12.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:12 smithi160 ceph-mon[40028]: osdmap e436: 8 total, 8 up, 8 in 2024-09-16T08:55:12.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:55:13.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:55:13.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:13 smithi169 ceph-mon[33465]: osdmap e437: 8 total, 8 up, 8 in 2024-09-16T08:55:13.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:13.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-81"} : dispatch 2024-09-16T08:55:13.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:55:13.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:13 smithi160 ceph-mon[30137]: osdmap e437: 8 total, 8 up, 8 in 2024-09-16T08:55:13.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:13.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-81"} : dispatch 2024-09-16T08:55:13.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:55:13.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:13 smithi160 ceph-mon[40028]: osdmap e437: 8 total, 8 up, 8 in 2024-09-16T08:55:13.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:13.951 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-81"} : dispatch 2024-09-16T08:55:14.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:14 smithi169 ceph-mon[33465]: pgmap v651: 228 pgs: 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:55:14.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:14 smithi169 ceph-mon[33465]: osdmap e438: 8 total, 8 up, 8 in 2024-09-16T08:55:14.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:14 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:14.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:14 smithi160 ceph-mon[40028]: pgmap v651: 228 pgs: 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:55:14.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:14 smithi160 ceph-mon[40028]: osdmap e438: 8 total, 8 up, 8 in 2024-09-16T08:55:14.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:14 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:14.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:14 smithi160 ceph-mon[30137]: pgmap v651: 228 pgs: 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:55:14.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:14 smithi160 ceph-mon[30137]: osdmap e438: 8 total, 8 up, 8 in 2024-09-16T08:55:14.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:14 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:15.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:55:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:55:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:55:15.581 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:55:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:55:15.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:15 smithi169 ceph-mon[33465]: osdmap e439: 8 total, 8 up, 8 in 2024-09-16T08:55:15.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:15 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:15.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:15 smithi160 ceph-mon[30137]: osdmap e439: 8 total, 8 up, 8 in 2024-09-16T08:55:15.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:15 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:15.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:15 smithi160 ceph-mon[40028]: osdmap e439: 8 total, 8 up, 8 in 2024-09-16T08:55:15.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:15 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:16 smithi169 ceph-mon[33465]: pgmap v654: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-16T08:55:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:16 smithi169 ceph-mon[33465]: osdmap e440: 8 total, 8 up, 8 in 2024-09-16T08:55:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:55:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:16 smithi160 ceph-mon[30137]: pgmap v654: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-16T08:55:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:16 smithi160 ceph-mon[30137]: osdmap e440: 8 total, 8 up, 8 in 2024-09-16T08:55:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:55:16.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:16.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:16 smithi160 ceph-mon[40028]: pgmap v654: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-16T08:55:16.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:16.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:16 smithi160 ceph-mon[40028]: osdmap e440: 8 total, 8 up, 8 in 2024-09-16T08:55:16.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:16.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:55:17.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:55:17.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:17 smithi169 ceph-mon[33465]: osdmap e441: 8 total, 8 up, 8 in 2024-09-16T08:55:17.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:17.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:55:17.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:17 smithi169 ceph-mon[33465]: osdmap e442: 8 total, 8 up, 8 in 2024-09-16T08:55:17.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-16T08:55:17.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:55:17.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:17 smithi160 ceph-mon[30137]: osdmap e441: 8 total, 8 up, 8 in 2024-09-16T08:55:17.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:17.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:55:17.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:17 smithi160 ceph-mon[30137]: osdmap e442: 8 total, 8 up, 8 in 2024-09-16T08:55:17.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-16T08:55:17.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:55:17.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:17 smithi160 ceph-mon[40028]: osdmap e441: 8 total, 8 up, 8 in 2024-09-16T08:55:17.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:17.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:55:17.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:17 smithi160 ceph-mon[40028]: osdmap e442: 8 total, 8 up, 8 in 2024-09-16T08:55:17.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-16T08:55:18.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:18 smithi169 ceph-mon[33465]: pgmap v657: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-16T08:55:18.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-16T08:55:18.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:18 smithi169 ceph-mon[33465]: osdmap e443: 8 total, 8 up, 8 in 2024-09-16T08:55:18.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:55:18.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:18 smithi160 ceph-mon[30137]: pgmap v657: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-16T08:55:18.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-16T08:55:18.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:18 smithi160 ceph-mon[30137]: osdmap e443: 8 total, 8 up, 8 in 2024-09-16T08:55:18.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:55:18.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:18 smithi160 ceph-mon[40028]: pgmap v657: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-16T08:55:18.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-16T08:55:18.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:18 smithi160 ceph-mon[40028]: osdmap e443: 8 total, 8 up, 8 in 2024-09-16T08:55:18.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:55:19.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:55:19.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:55:19.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:55:19.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:55:20.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:55:20.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:55:20.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:20 smithi160 ceph-mon[30137]: pgmap v660: 228 pgs: 228 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:55:20.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:20 smithi160 ceph-mon[30137]: osdmap e444: 8 total, 8 up, 8 in 2024-09-16T08:55:20.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:20 smithi160 ceph-mon[40028]: pgmap v660: 228 pgs: 228 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:55:20.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:20 smithi160 ceph-mon[40028]: osdmap e444: 8 total, 8 up, 8 in 2024-09-16T08:55:21.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:20 smithi169 ceph-mon[33465]: pgmap v660: 228 pgs: 228 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:55:21.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:20 smithi169 ceph-mon[33465]: osdmap e444: 8 total, 8 up, 8 in 2024-09-16T08:55:21.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:21 smithi160 ceph-mon[30137]: osdmap e445: 8 total, 8 up, 8 in 2024-09-16T08:55:21.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:21 smithi160 ceph-mon[40028]: osdmap e445: 8 total, 8 up, 8 in 2024-09-16T08:55:22.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:21 smithi169 ceph-mon[33465]: osdmap e445: 8 total, 8 up, 8 in 2024-09-16T08:55:22.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:22 smithi160 ceph-mon[30137]: pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:55:22.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:22 smithi160 ceph-mon[30137]: osdmap e446: 8 total, 8 up, 8 in 2024-09-16T08:55:22.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:22 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:22.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:22 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-83"} : dispatch 2024-09-16T08:55:22.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:22 smithi160 ceph-mon[40028]: pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:55:22.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:22 smithi160 ceph-mon[40028]: osdmap e446: 8 total, 8 up, 8 in 2024-09-16T08:55:22.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:22 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:22.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:22 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-83"} : dispatch 2024-09-16T08:55:23.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:22 smithi169 ceph-mon[33465]: pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:55:23.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:22 smithi169 ceph-mon[33465]: osdmap e446: 8 total, 8 up, 8 in 2024-09-16T08:55:23.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:22 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:23.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:22 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-83"} : dispatch 2024-09-16T08:55:23.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:23 smithi160 ceph-mon[30137]: osdmap e447: 8 total, 8 up, 8 in 2024-09-16T08:55:23.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:23 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:55:23.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:23 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:55:23.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:23 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:55:23.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:23 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:55:23.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:23 smithi160 ceph-mon[40028]: osdmap e447: 8 total, 8 up, 8 in 2024-09-16T08:55:23.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:23 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:55:23.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:23 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:55:23.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:23 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:55:23.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:23 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:55:24.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:23 smithi169 ceph-mon[33465]: osdmap e447: 8 total, 8 up, 8 in 2024-09-16T08:55:24.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:23 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:55:24.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:23 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:55:24.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:23 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:55:24.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:23 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:55:24.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:24 smithi160 ceph-mon[30137]: pgmap v666: 196 pgs: 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:55:24.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:24 smithi160 ceph-mon[30137]: osdmap e448: 8 total, 8 up, 8 in 2024-09-16T08:55:24.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:24.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:24 smithi160 ceph-mon[40028]: pgmap v666: 196 pgs: 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:55:24.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:24 smithi160 ceph-mon[40028]: osdmap e448: 8 total, 8 up, 8 in 2024-09-16T08:55:24.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:25.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:24 smithi169 ceph-mon[33465]: pgmap v666: 196 pgs: 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:55:25.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:24 smithi169 ceph-mon[33465]: osdmap e448: 8 total, 8 up, 8 in 2024-09-16T08:55:25.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:25.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:55:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:55:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:55:25.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:55:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:55:25.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:25.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:25 smithi160 ceph-mon[30137]: osdmap e449: 8 total, 8 up, 8 in 2024-09-16T08:55:25.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:25.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:55:25.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:25.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:25 smithi160 ceph-mon[40028]: osdmap e449: 8 total, 8 up, 8 in 2024-09-16T08:55:25.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:25.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:55:26.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:26.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:25 smithi169 ceph-mon[33465]: osdmap e449: 8 total, 8 up, 8 in 2024-09-16T08:55:26.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:26.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:55:26.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:26.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:26 smithi160 ceph-mon[40028]: pgmap v669: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-16T08:55:26.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:55:26.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:26 smithi160 ceph-mon[40028]: osdmap e450: 8 total, 8 up, 8 in 2024-09-16T08:55:26.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:26.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:26.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:26 smithi160 ceph-mon[30137]: pgmap v669: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-16T08:55:26.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:55:26.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:26 smithi160 ceph-mon[30137]: osdmap e450: 8 total, 8 up, 8 in 2024-09-16T08:55:26.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:27.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:27.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:26 smithi169 ceph-mon[33465]: pgmap v669: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-16T08:55:27.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:55:27.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:26 smithi169 ceph-mon[33465]: osdmap e450: 8 total, 8 up, 8 in 2024-09-16T08:55:27.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:27.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:55:27.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:27 smithi160 ceph-mon[30137]: osdmap e451: 8 total, 8 up, 8 in 2024-09-16T08:55:27.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-16T08:55:27.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-16T08:55:27.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:55:27.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:27 smithi160 ceph-mon[40028]: osdmap e451: 8 total, 8 up, 8 in 2024-09-16T08:55:27.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-16T08:55:27.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-16T08:55:28.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_tier","val": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:55:28.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:27 smithi169 ceph-mon[33465]: osdmap e451: 8 total, 8 up, 8 in 2024-09-16T08:55:28.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-16T08:55:28.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-16T08:55:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:28 smithi160 ceph-mon[30137]: pgmap v672: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-16T08:55:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:28 smithi160 ceph-mon[30137]: osdmap e452: 8 total, 8 up, 8 in 2024-09-16T08:55:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:55:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:55:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:28 smithi160 ceph-mon[30137]: osdmap e453: 8 total, 8 up, 8 in 2024-09-16T08:55:28.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:28 smithi160 ceph-mon[40028]: pgmap v672: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-16T08:55:28.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:28 smithi160 ceph-mon[40028]: osdmap e452: 8 total, 8 up, 8 in 2024-09-16T08:55:28.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:55:28.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:55:28.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:28 smithi160 ceph-mon[40028]: osdmap e453: 8 total, 8 up, 8 in 2024-09-16T08:55:29.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:28 smithi169 ceph-mon[33465]: pgmap v672: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-16T08:55:29.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:28 smithi169 ceph-mon[33465]: osdmap e452: 8 total, 8 up, 8 in 2024-09-16T08:55:29.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:55:29.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:55:29.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:28 smithi169 ceph-mon[33465]: osdmap e453: 8 total, 8 up, 8 in 2024-09-16T08:55:29.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:29 smithi160 ceph-mon[30137]: pgmap v675: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-16T08:55:29.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:29 smithi160 ceph-mon[40028]: pgmap v675: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-16T08:55:30.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:29 smithi169 ceph-mon[33465]: pgmap v675: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-16T08:55:30.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:30 smithi160 ceph-mon[30137]: osdmap e454: 8 total, 8 up, 8 in 2024-09-16T08:55:30.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:30 smithi160 ceph-mon[40028]: osdmap e454: 8 total, 8 up, 8 in 2024-09-16T08:55:31.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:30 smithi169 ceph-mon[33465]: osdmap e454: 8 total, 8 up, 8 in 2024-09-16T08:55:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:31 smithi160 ceph-mon[30137]: osdmap e455: 8 total, 8 up, 8 in 2024-09-16T08:55:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-85"} : dispatch 2024-09-16T08:55:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:31 smithi160 ceph-mon[30137]: pgmap v678: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-16T08:55:31.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:31 smithi160 ceph-mon[40028]: osdmap e455: 8 total, 8 up, 8 in 2024-09-16T08:55:31.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-85"} : dispatch 2024-09-16T08:55:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:31 smithi160 ceph-mon[40028]: pgmap v678: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-16T08:55:32.001 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:31 smithi169 ceph-mon[33465]: osdmap e455: 8 total, 8 up, 8 in 2024-09-16T08:55:32.001 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:32.001 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-85"} : dispatch 2024-09-16T08:55:32.001 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:31 smithi169 ceph-mon[33465]: pgmap v678: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-16T08:55:32.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:32 smithi160 ceph-mon[30137]: osdmap e456: 8 total, 8 up, 8 in 2024-09-16T08:55:32.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:32 smithi160 ceph-mon[40028]: osdmap e456: 8 total, 8 up, 8 in 2024-09-16T08:55:33.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:32 smithi169 ceph-mon[33465]: osdmap e456: 8 total, 8 up, 8 in 2024-09-16T08:55:33.698 INFO:tasks.workunit.client.0.smithi160.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18076 ms) 2024-09-16T08:55:33.698 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-09-16T08:55:33.698 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22790 ms) 2024-09-16T08:55:33.698 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-09-16T08:55:33.698 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3032 ms) 2024-09-16T08:55:33.698 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-09-16T08:55:33.698 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:55:33.698 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3007 ms) 2024-09-16T08:55:33.698 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-09-16T08:55:33.698 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3003 ms) 2024-09-16T08:55:33.698 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-09-16T08:55:33.699 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:55:33.699 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3006 ms) 2024-09-16T08:55:33.699 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-09-16T08:55:33.699 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:55:33.699 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4021 ms) 2024-09-16T08:55:33.699 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-09-16T08:55:33.699 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:55:33.699 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (39883 ms) 2024-09-16T08:55:33.699 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-09-16T08:55:33.699 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:55:33.699 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (17004 ms) 2024-09-16T08:55:33.699 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-09-16T08:55:33.699 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:55:33.700 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4029 ms) 2024-09-16T08:55:33.700 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-09-16T08:55:33.700 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:55:33.700 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3020 ms) 2024-09-16T08:55:33.700 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-09-16T08:55:33.700 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:55:33.700 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (25074 ms) 2024-09-16T08:55:33.700 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-09-16T08:55:33.700 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:55:33.700 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15037 ms) 2024-09-16T08:55:33.700 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-09-16T08:55:33.700 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:55:33.700 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5049 ms) 2024-09-16T08:55:33.701 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-09-16T08:55:33.701 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:55:33.701 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4027 ms) 2024-09-16T08:55:33.701 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-09-16T08:55:33.701 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:55:33.701 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: waiting for scrubs... 2024-09-16T08:55:33.701 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: done waiting 2024-09-16T08:55:33.701 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25057 ms) 2024-09-16T08:55:33.701 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-09-16T08:55:33.701 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:55:33.701 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10078 ms) 2024-09-16T08:55:33.701 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-09-16T08:55:33.701 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:55:33.701 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9053 ms) 2024-09-16T08:55:33.702 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-09-16T08:55:33.702 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:55:33.702 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9066 ms) 2024-09-16T08:55:33.702 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-09-16T08:55:34.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:33 smithi169 ceph-mon[33465]: osdmap e457: 8 total, 8 up, 8 in 2024-09-16T08:55:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:33 smithi169 ceph-mon[33465]: pgmap v681: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:55:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:33 smithi169 ceph-mon[33465]: osdmap e458: 8 total, 8 up, 8 in 2024-09-16T08:55:34.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:33 smithi160 ceph-mon[30137]: osdmap e457: 8 total, 8 up, 8 in 2024-09-16T08:55:34.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:34.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:33 smithi160 ceph-mon[30137]: pgmap v681: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:55:34.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:34.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:33 smithi160 ceph-mon[30137]: osdmap e458: 8 total, 8 up, 8 in 2024-09-16T08:55:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:33 smithi160 ceph-mon[40028]: osdmap e457: 8 total, 8 up, 8 in 2024-09-16T08:55:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:33 smithi160 ceph-mon[40028]: pgmap v681: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:55:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:34.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:33 smithi160 ceph-mon[40028]: osdmap e458: 8 total, 8 up, 8 in 2024-09-16T08:55:34.989 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:34.989 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:34 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:34.989 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:55:34.989 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:34 smithi160 ceph-mon[30137]: osdmap e459: 8 total, 8 up, 8 in 2024-09-16T08:55:34.989 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:55:34.989 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:34.989 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:34 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:34.989 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:55:34.989 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:34 smithi160 ceph-mon[40028]: osdmap e459: 8 total, 8 up, 8 in 2024-09-16T08:55:34.990 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:55:35.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:35.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:34 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:35.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:55:35.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:34 smithi169 ceph-mon[33465]: osdmap e459: 8 total, 8 up, 8 in 2024-09-16T08:55:35.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:55:35.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:55:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:55:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:55:35.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:55:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:55:36.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:35 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:36.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:35 smithi169 ceph-mon[33465]: pgmap v684: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-16T08:55:36.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:55:36.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:35 smithi169 ceph-mon[33465]: osdmap e460: 8 total, 8 up, 8 in 2024-09-16T08:55:36.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_tier","val": "test-rados-api-smithi160-76079-89-test-flush"} : dispatch 2024-09-16T08:55:36.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:35 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:36.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:35 smithi160 ceph-mon[30137]: pgmap v684: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-16T08:55:36.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:55:36.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:35 smithi160 ceph-mon[30137]: osdmap e460: 8 total, 8 up, 8 in 2024-09-16T08:55:36.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_tier","val": "test-rados-api-smithi160-76079-89-test-flush"} : dispatch 2024-09-16T08:55:36.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:35 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:36.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:35 smithi160 ceph-mon[40028]: pgmap v684: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-16T08:55:36.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:55:36.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:35 smithi160 ceph-mon[40028]: osdmap e460: 8 total, 8 up, 8 in 2024-09-16T08:55:36.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_tier","val": "test-rados-api-smithi160-76079-89-test-flush"} : dispatch 2024-09-16T08:55:38.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_tier","val": "test-rados-api-smithi160-76079-89-test-flush"}]': finished 2024-09-16T08:55:38.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:37 smithi169 ceph-mon[33465]: osdmap e461: 8 total, 8 up, 8 in 2024-09-16T08:55:38.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-16T08:55:38.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:37 smithi169 ceph-mon[33465]: pgmap v687: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-09-16T08:55:38.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_tier","val": "test-rados-api-smithi160-76079-89-test-flush"}]': finished 2024-09-16T08:55:38.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:37 smithi160 ceph-mon[30137]: osdmap e461: 8 total, 8 up, 8 in 2024-09-16T08:55:38.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-16T08:55:38.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:37 smithi160 ceph-mon[30137]: pgmap v687: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-09-16T08:55:38.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_tier","val": "test-rados-api-smithi160-76079-89-test-flush"}]': finished 2024-09-16T08:55:38.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:37 smithi160 ceph-mon[40028]: osdmap e461: 8 total, 8 up, 8 in 2024-09-16T08:55:38.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-16T08:55:38.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:37 smithi160 ceph-mon[40028]: pgmap v687: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-09-16T08:55:39.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-16T08:55:39.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:38 smithi169 ceph-mon[33465]: osdmap e462: 8 total, 8 up, 8 in 2024-09-16T08:55:39.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:55:39.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-16T08:55:39.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:38 smithi160 ceph-mon[30137]: osdmap e462: 8 total, 8 up, 8 in 2024-09-16T08:55:39.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:55:39.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-16T08:55:39.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:38 smithi160 ceph-mon[40028]: osdmap e462: 8 total, 8 up, 8 in 2024-09-16T08:55:39.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:55:40.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:55:40.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:39 smithi169 ceph-mon[33465]: osdmap e463: 8 total, 8 up, 8 in 2024-09-16T08:55:40.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:39 smithi169 ceph-mon[33465]: pgmap v690: 260 pgs: 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-16T08:55:40.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:55:40.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:39 smithi160 ceph-mon[40028]: osdmap e463: 8 total, 8 up, 8 in 2024-09-16T08:55:40.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:39 smithi160 ceph-mon[40028]: pgmap v690: 260 pgs: 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-16T08:55:40.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:55:40.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:39 smithi160 ceph-mon[30137]: osdmap e463: 8 total, 8 up, 8 in 2024-09-16T08:55:40.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:39 smithi160 ceph-mon[30137]: pgmap v690: 260 pgs: 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-16T08:55:41.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:40 smithi169 ceph-mon[33465]: osdmap e464: 8 total, 8 up, 8 in 2024-09-16T08:55:41.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:41.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-87"} : dispatch 2024-09-16T08:55:41.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:40 smithi160 ceph-mon[30137]: osdmap e464: 8 total, 8 up, 8 in 2024-09-16T08:55:41.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:41.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-87"} : dispatch 2024-09-16T08:55:41.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:40 smithi160 ceph-mon[40028]: osdmap e464: 8 total, 8 up, 8 in 2024-09-16T08:55:41.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:41.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-87"} : dispatch 2024-09-16T08:55:42.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:41 smithi169 ceph-mon[33465]: osdmap e465: 8 total, 8 up, 8 in 2024-09-16T08:55:42.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:41 smithi169 ceph-mon[33465]: pgmap v693: 196 pgs: 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:55:42.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:41 smithi169 ceph-mon[33465]: osdmap e466: 8 total, 8 up, 8 in 2024-09-16T08:55:42.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:42.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:41 smithi160 ceph-mon[30137]: osdmap e465: 8 total, 8 up, 8 in 2024-09-16T08:55:42.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:41 smithi160 ceph-mon[30137]: pgmap v693: 196 pgs: 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:55:42.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:41 smithi160 ceph-mon[30137]: osdmap e466: 8 total, 8 up, 8 in 2024-09-16T08:55:42.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:42.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:41 smithi160 ceph-mon[40028]: osdmap e465: 8 total, 8 up, 8 in 2024-09-16T08:55:42.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:41 smithi160 ceph-mon[40028]: pgmap v693: 196 pgs: 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:55:42.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:41 smithi160 ceph-mon[40028]: osdmap e466: 8 total, 8 up, 8 in 2024-09-16T08:55:42.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:43.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:42 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:43.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:43.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:42 smithi169 ceph-mon[33465]: osdmap e467: 8 total, 8 up, 8 in 2024-09-16T08:55:43.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:43.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:42 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:43.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:43.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:42 smithi160 ceph-mon[30137]: osdmap e467: 8 total, 8 up, 8 in 2024-09-16T08:55:43.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:43.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:42 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:43.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:43.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:42 smithi160 ceph-mon[40028]: osdmap e467: 8 total, 8 up, 8 in 2024-09-16T08:55:43.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:44.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:55:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:43 smithi169 ceph-mon[33465]: pgmap v696: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:55:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:55:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:43 smithi169 ceph-mon[33465]: osdmap e468: 8 total, 8 up, 8 in 2024-09-16T08:55:44.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:55:44.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:43 smithi160 ceph-mon[30137]: pgmap v696: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:55:44.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:55:44.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:43 smithi160 ceph-mon[30137]: osdmap e468: 8 total, 8 up, 8 in 2024-09-16T08:55:44.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:55:44.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:43 smithi160 ceph-mon[40028]: pgmap v696: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:55:44.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:55:44.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:43 smithi160 ceph-mon[40028]: osdmap e468: 8 total, 8 up, 8 in 2024-09-16T08:55:45.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:55:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:55:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:55:45.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:55:45 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:55:46.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:45 smithi169 ceph-mon[33465]: osdmap e469: 8 total, 8 up, 8 in 2024-09-16T08:55:46.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:45 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:46.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:45 smithi169 ceph-mon[33465]: pgmap v699: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 2.2 KiB/s wr, 183 op/s 2024-09-16T08:55:46.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:45 smithi160 ceph-mon[30137]: osdmap e469: 8 total, 8 up, 8 in 2024-09-16T08:55:46.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:45 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:46.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:45 smithi160 ceph-mon[30137]: pgmap v699: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 2.2 KiB/s wr, 183 op/s 2024-09-16T08:55:46.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:45 smithi160 ceph-mon[40028]: osdmap e469: 8 total, 8 up, 8 in 2024-09-16T08:55:46.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:45 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:46.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:45 smithi160 ceph-mon[40028]: pgmap v699: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 2.2 KiB/s wr, 183 op/s 2024-09-16T08:55:47.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:46 smithi169 ceph-mon[33465]: osdmap e470: 8 total, 8 up, 8 in 2024-09-16T08:55:47.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:47.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-90"} : dispatch 2024-09-16T08:55:47.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:46 smithi160 ceph-mon[30137]: osdmap e470: 8 total, 8 up, 8 in 2024-09-16T08:55:47.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:47.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-90"} : dispatch 2024-09-16T08:55:47.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:46 smithi160 ceph-mon[40028]: osdmap e470: 8 total, 8 up, 8 in 2024-09-16T08:55:47.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:47.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-90"} : dispatch 2024-09-16T08:55:48.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:47 smithi169 ceph-mon[33465]: osdmap e471: 8 total, 8 up, 8 in 2024-09-16T08:55:48.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:47 smithi169 ceph-mon[33465]: pgmap v702: 196 pgs: 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 255 B/s wr, 180 op/s 2024-09-16T08:55:48.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:47 smithi160 ceph-mon[30137]: osdmap e471: 8 total, 8 up, 8 in 2024-09-16T08:55:48.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:47 smithi160 ceph-mon[30137]: pgmap v702: 196 pgs: 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 255 B/s wr, 180 op/s 2024-09-16T08:55:48.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:47 smithi160 ceph-mon[40028]: osdmap e471: 8 total, 8 up, 8 in 2024-09-16T08:55:48.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:47 smithi160 ceph-mon[40028]: pgmap v702: 196 pgs: 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 255 B/s wr, 180 op/s 2024-09-16T08:55:49.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:48 smithi169 ceph-mon[33465]: osdmap e472: 8 total, 8 up, 8 in 2024-09-16T08:55:49.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:49.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:48 smithi160 ceph-mon[30137]: osdmap e472: 8 total, 8 up, 8 in 2024-09-16T08:55:49.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:49.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:48 smithi160 ceph-mon[40028]: osdmap e472: 8 total, 8 up, 8 in 2024-09-16T08:55:49.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:50.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:50.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:49 smithi169 ceph-mon[33465]: osdmap e473: 8 total, 8 up, 8 in 2024-09-16T08:55:50.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:50.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:55:50.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:49 smithi169 ceph-mon[33465]: pgmap v705: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:55:50.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:50.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:49 smithi160 ceph-mon[30137]: osdmap e473: 8 total, 8 up, 8 in 2024-09-16T08:55:50.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:50.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:55:50.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:49 smithi160 ceph-mon[30137]: pgmap v705: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:55:50.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:50.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:49 smithi160 ceph-mon[40028]: osdmap e473: 8 total, 8 up, 8 in 2024-09-16T08:55:50.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:50.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:55:50.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:49 smithi160 ceph-mon[40028]: pgmap v705: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:55:51.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:50 smithi169 ceph-mon[33465]: osdmap e474: 8 total, 8 up, 8 in 2024-09-16T08:55:51.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:50 smithi160 ceph-mon[30137]: osdmap e474: 8 total, 8 up, 8 in 2024-09-16T08:55:51.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:50 smithi160 ceph-mon[40028]: osdmap e474: 8 total, 8 up, 8 in 2024-09-16T08:55:52.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:51 smithi169 ceph-mon[33465]: osdmap e475: 8 total, 8 up, 8 in 2024-09-16T08:55:52.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:52.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-92"} : dispatch 2024-09-16T08:55:52.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:51 smithi169 ceph-mon[33465]: pgmap v708: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-09-16T08:55:52.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:51 smithi160 ceph-mon[40028]: osdmap e475: 8 total, 8 up, 8 in 2024-09-16T08:55:52.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:52.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-92"} : dispatch 2024-09-16T08:55:52.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:51 smithi160 ceph-mon[40028]: pgmap v708: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-09-16T08:55:52.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:51 smithi160 ceph-mon[30137]: osdmap e475: 8 total, 8 up, 8 in 2024-09-16T08:55:52.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:55:52.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-92"} : dispatch 2024-09-16T08:55:52.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:51 smithi160 ceph-mon[30137]: pgmap v708: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-09-16T08:55:53.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:52 smithi169 ceph-mon[33465]: osdmap e476: 8 total, 8 up, 8 in 2024-09-16T08:55:53.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:52 smithi160 ceph-mon[30137]: osdmap e476: 8 total, 8 up, 8 in 2024-09-16T08:55:53.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:52 smithi160 ceph-mon[40028]: osdmap e476: 8 total, 8 up, 8 in 2024-09-16T08:55:54.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:53 smithi169 ceph-mon[33465]: osdmap e477: 8 total, 8 up, 8 in 2024-09-16T08:55:54.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:54.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:53 smithi169 ceph-mon[33465]: pgmap v711: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:55:54.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:53 smithi160 ceph-mon[30137]: osdmap e477: 8 total, 8 up, 8 in 2024-09-16T08:55:54.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:54.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:53 smithi160 ceph-mon[30137]: pgmap v711: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:55:54.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:53 smithi160 ceph-mon[40028]: osdmap e477: 8 total, 8 up, 8 in 2024-09-16T08:55:54.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:55:54.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:53 smithi160 ceph-mon[40028]: pgmap v711: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:55:55.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:54 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:55.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:55.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:54 smithi169 ceph-mon[33465]: osdmap e478: 8 total, 8 up, 8 in 2024-09-16T08:55:55.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:54 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:54 smithi160 ceph-mon[30137]: osdmap e478: 8 total, 8 up, 8 in 2024-09-16T08:55:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:55.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:54 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:55.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:55:55.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:54 smithi160 ceph-mon[40028]: osdmap e478: 8 total, 8 up, 8 in 2024-09-16T08:55:55.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:55:55.199 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:55:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:55:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:55:55.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:55:55 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:55:56.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:55 smithi169 ceph-mon[33465]: osdmap e479: 8 total, 8 up, 8 in 2024-09-16T08:55:56.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:55 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:56.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:55 smithi169 ceph-mon[33465]: pgmap v714: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:55:56.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:55 smithi160 ceph-mon[30137]: osdmap e479: 8 total, 8 up, 8 in 2024-09-16T08:55:56.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:55 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:56.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:55 smithi160 ceph-mon[30137]: pgmap v714: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:55:56.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:55 smithi160 ceph-mon[40028]: osdmap e479: 8 total, 8 up, 8 in 2024-09-16T08:55:56.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:55 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:55:56.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:55 smithi160 ceph-mon[40028]: pgmap v714: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:55:57.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:56 smithi169 ceph-mon[33465]: osdmap e480: 8 total, 8 up, 8 in 2024-09-16T08:55:57.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:56 smithi160 ceph-mon[30137]: osdmap e480: 8 total, 8 up, 8 in 2024-09-16T08:55:57.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:56 smithi160 ceph-mon[40028]: osdmap e480: 8 total, 8 up, 8 in 2024-09-16T08:55:58.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:57 smithi160 ceph-mon[30137]: pgmap v716: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-09-16T08:55:58.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:57 smithi160 ceph-mon[40028]: pgmap v716: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-09-16T08:55:58.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:57 smithi169 ceph-mon[33465]: pgmap v716: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-09-16T08:55:59.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:55:59 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:59.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:55:59 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:55:59.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:55:59 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:00.599 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:00 smithi160 ceph-mon[40028]: pgmap v717: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T08:56:00.600 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:00 smithi160 ceph-mon[30137]: pgmap v717: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T08:56:00.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:00 smithi169 ceph-mon[33465]: pgmap v717: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T08:56:02.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:02 smithi169 ceph-mon[33465]: pgmap v718: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-16T08:56:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:02 smithi160 ceph-mon[30137]: pgmap v718: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-16T08:56:02.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:02 smithi160 ceph-mon[40028]: pgmap v718: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-16T08:56:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:04 smithi169 ceph-mon[33465]: pgmap v719: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 243 B/s wr, 2 op/s 2024-09-16T08:56:04.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:56:04.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:04 smithi160 ceph-mon[30137]: pgmap v719: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 243 B/s wr, 2 op/s 2024-09-16T08:56:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:56:04.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:04 smithi160 ceph-mon[40028]: pgmap v719: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 243 B/s wr, 2 op/s 2024-09-16T08:56:04.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:56:05.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:56:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:56:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:56:05.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:56:05 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:56:06.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:06 smithi169 ceph-mon[33465]: pgmap v720: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-16T08:56:06.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:06 smithi160 ceph-mon[30137]: pgmap v720: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-16T08:56:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:06 smithi160 ceph-mon[40028]: pgmap v720: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-16T08:56:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:07.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:07 smithi169 ceph-mon[33465]: osdmap e481: 8 total, 8 up, 8 in 2024-09-16T08:56:07.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:07 smithi160 ceph-mon[30137]: osdmap e481: 8 total, 8 up, 8 in 2024-09-16T08:56:07.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:07 smithi160 ceph-mon[40028]: osdmap e481: 8 total, 8 up, 8 in 2024-09-16T08:56:08.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:08 smithi169 ceph-mon[33465]: pgmap v722: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-16T08:56:08.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:08 smithi160 ceph-mon[30137]: pgmap v722: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-16T08:56:08.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:08 smithi160 ceph-mon[40028]: pgmap v722: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-16T08:56:10.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:10 smithi169 ceph-mon[33465]: pgmap v723: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:10.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:10 smithi160 ceph-mon[30137]: pgmap v723: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:10.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:10 smithi160 ceph-mon[40028]: pgmap v723: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:12.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:12 smithi169 ceph-mon[33465]: pgmap v724: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:12.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:12 smithi160 ceph-mon[30137]: pgmap v724: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:12.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:12 smithi160 ceph-mon[40028]: pgmap v724: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:14.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:14 smithi169 ceph-mon[33465]: pgmap v725: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:14.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:14 smithi169 ceph-mon[33465]: osdmap e482: 8 total, 8 up, 8 in 2024-09-16T08:56:14.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:14 smithi160 ceph-mon[30137]: pgmap v725: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:14 smithi160 ceph-mon[30137]: osdmap e482: 8 total, 8 up, 8 in 2024-09-16T08:56:14.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:14 smithi160 ceph-mon[40028]: pgmap v725: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:14.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:14 smithi160 ceph-mon[40028]: osdmap e482: 8 total, 8 up, 8 in 2024-09-16T08:56:15.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:56:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:56:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:56:15.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:56:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:56:16.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:16 smithi169 ceph-mon[33465]: pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:16.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:16 smithi160 ceph-mon[30137]: pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:16 smithi160 ceph-mon[40028]: pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:17.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:56:17.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-94"} : dispatch 2024-09-16T08:56:17.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:56:17.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-94"} : dispatch 2024-09-16T08:56:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:56:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-94"} : dispatch 2024-09-16T08:56:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:18 smithi169 ceph-mon[33465]: pgmap v728: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:18 smithi169 ceph-mon[33465]: osdmap e483: 8 total, 8 up, 8 in 2024-09-16T08:56:18.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:18 smithi160 ceph-mon[30137]: pgmap v728: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:18.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:18 smithi160 ceph-mon[30137]: osdmap e483: 8 total, 8 up, 8 in 2024-09-16T08:56:18.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:18 smithi160 ceph-mon[40028]: pgmap v728: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:18.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:18 smithi160 ceph-mon[40028]: osdmap e483: 8 total, 8 up, 8 in 2024-09-16T08:56:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:19 smithi169 ceph-mon[33465]: osdmap e484: 8 total, 8 up, 8 in 2024-09-16T08:56:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:56:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:56:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:19 smithi160 ceph-mon[40028]: osdmap e484: 8 total, 8 up, 8 in 2024-09-16T08:56:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:56:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:56:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:19 smithi160 ceph-mon[30137]: osdmap e484: 8 total, 8 up, 8 in 2024-09-16T08:56:19.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:56:19.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:56:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:20 smithi169 ceph-mon[33465]: pgmap v731: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:56:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:20 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:56:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:20 smithi169 ceph-mon[33465]: osdmap e485: 8 total, 8 up, 8 in 2024-09-16T08:56:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:56:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:20 smithi160 ceph-mon[40028]: pgmap v731: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:56:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:20 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:56:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:20 smithi160 ceph-mon[40028]: osdmap e485: 8 total, 8 up, 8 in 2024-09-16T08:56:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:56:20.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:20 smithi160 ceph-mon[30137]: pgmap v731: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:56:20.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:20 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:20.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:56:20.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:20 smithi160 ceph-mon[30137]: osdmap e485: 8 total, 8 up, 8 in 2024-09-16T08:56:20.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:56:21.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:21 smithi169 ceph-mon[33465]: osdmap e486: 8 total, 8 up, 8 in 2024-09-16T08:56:21.701 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:21 smithi160 ceph-mon[30137]: osdmap e486: 8 total, 8 up, 8 in 2024-09-16T08:56:21.702 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:21 smithi160 ceph-mon[40028]: osdmap e486: 8 total, 8 up, 8 in 2024-09-16T08:56:22.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:22 smithi169 ceph-mon[33465]: pgmap v734: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:56:22.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:22 smithi160 ceph-mon[30137]: pgmap v734: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:56:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:22 smithi160 ceph-mon[40028]: pgmap v734: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:56:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[30137]: pgmap v735: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 866 B/s rd, 173 B/s wr, 1 op/s 2024-09-16T08:56:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:56:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:56:24.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:56:24.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:56:24.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:56:24.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:56:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[40028]: pgmap v735: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 866 B/s rd, 173 B/s wr, 1 op/s 2024-09-16T08:56:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:56:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:56:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:56:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:56:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:56:24.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:56:24.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:24 smithi169 ceph-mon[33465]: pgmap v735: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 866 B/s rd, 173 B/s wr, 1 op/s 2024-09-16T08:56:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:24 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:56:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:24 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:24 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:24 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:24 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:24 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:24 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:24 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:24 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T08:56:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:24 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:56:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:24 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:56:24.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:24 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:56:24.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:24 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:56:24.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:24 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:56:25.357 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:56:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:56:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:56:25.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:25 smithi169 ceph-mon[33465]: Adjusting osd_memory_target on smithi169 to 3248M 2024-09-16T08:56:25.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:25 smithi169 ceph-mon[33465]: Adjusting osd_memory_target on smithi160 to 3248M 2024-09-16T08:56:25.600 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:56:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:56:25.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:25 smithi160 ceph-mon[30137]: Adjusting osd_memory_target on smithi169 to 3248M 2024-09-16T08:56:25.724 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:25 smithi160 ceph-mon[30137]: Adjusting osd_memory_target on smithi160 to 3248M 2024-09-16T08:56:25.724 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:25 smithi160 ceph-mon[40028]: Adjusting osd_memory_target on smithi169 to 3248M 2024-09-16T08:56:25.724 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:25 smithi160 ceph-mon[40028]: Adjusting osd_memory_target on smithi160 to 3248M 2024-09-16T08:56:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:26 smithi160 ceph-mon[30137]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 889 B/s wr, 3 op/s 2024-09-16T08:56:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:26 smithi160 ceph-mon[40028]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 889 B/s wr, 3 op/s 2024-09-16T08:56:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:26.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:26 smithi169 ceph-mon[33465]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 889 B/s wr, 3 op/s 2024-09-16T08:56:26.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:28 smithi160 ceph-mon[30137]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-09-16T08:56:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:28 smithi160 ceph-mon[40028]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-09-16T08:56:28.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:28 smithi169 ceph-mon[33465]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-09-16T08:56:29.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:29 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:29 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:29.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:29 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:30 smithi160 ceph-mon[30137]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 517 B/s wr, 2 op/s 2024-09-16T08:56:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:30 smithi160 ceph-mon[40028]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 517 B/s wr, 2 op/s 2024-09-16T08:56:30.698 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 08:56:30 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-1[49686]: 2024-09-16T08:56:30.364+0000 7f66ef652640 -1 snap_mapper.add_oid found existing snaps mapped on 108:8ae9288d:test-rados-api-smithi160-76079-97::foo:21, removing 2024-09-16T08:56:30.699 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 08:56:30 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-3[61676]: 2024-09-16T08:56:30.364+0000 7f72ba4c3640 -1 snap_mapper.add_oid found existing snaps mapped on 108:8ae9288d:test-rados-api-smithi160-76079-97::foo:21, removing 2024-09-16T08:56:30.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:30 smithi169 ceph-mon[33465]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 517 B/s wr, 2 op/s 2024-09-16T08:56:30.850 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 08:56:30 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T08:56:30.365+0000 7f51da947640 -1 snap_mapper.add_oid found existing snaps mapped on 108:8ae9288d:test-rados-api-smithi160-76079-97::foo:21, removing 2024-09-16T08:56:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:56:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-96"} : dispatch 2024-09-16T08:56:31.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:56:31.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-96"} : dispatch 2024-09-16T08:56:31.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:56:31.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-96"} : dispatch 2024-09-16T08:56:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:32 smithi160 ceph-mon[30137]: pgmap v739: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 939 B/s rd, 469 B/s wr, 1 op/s 2024-09-16T08:56:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:32 smithi160 ceph-mon[30137]: osdmap e487: 8 total, 8 up, 8 in 2024-09-16T08:56:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:32 smithi160 ceph-mon[40028]: pgmap v739: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 939 B/s rd, 469 B/s wr, 1 op/s 2024-09-16T08:56:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:32 smithi160 ceph-mon[40028]: osdmap e487: 8 total, 8 up, 8 in 2024-09-16T08:56:32.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:32 smithi169 ceph-mon[33465]: pgmap v739: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 939 B/s rd, 469 B/s wr, 1 op/s 2024-09-16T08:56:32.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:32 smithi169 ceph-mon[33465]: osdmap e487: 8 total, 8 up, 8 in 2024-09-16T08:56:33.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:33 smithi160 ceph-mon[40028]: osdmap e488: 8 total, 8 up, 8 in 2024-09-16T08:56:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:56:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:33 smithi160 ceph-mon[30137]: osdmap e488: 8 total, 8 up, 8 in 2024-09-16T08:56:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:56:33.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:33 smithi169 ceph-mon[33465]: osdmap e488: 8 total, 8 up, 8 in 2024-09-16T08:56:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:56:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:34 smithi160 ceph-mon[30137]: pgmap v742: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-16T08:56:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:56:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:34 smithi160 ceph-mon[30137]: osdmap e489: 8 total, 8 up, 8 in 2024-09-16T08:56:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:56:34.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:56:34.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:34 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:34 smithi160 ceph-mon[40028]: pgmap v742: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-16T08:56:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:56:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:34 smithi160 ceph-mon[40028]: osdmap e489: 8 total, 8 up, 8 in 2024-09-16T08:56:34.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:56:34.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:56:34.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:34 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:34.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:34 smithi169 ceph-mon[33465]: pgmap v742: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-16T08:56:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:56:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:34 smithi169 ceph-mon[33465]: osdmap e489: 8 total, 8 up, 8 in 2024-09-16T08:56:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:56:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:56:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:34 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:35.408 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:56:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:56:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:56:35.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:56:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:56:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:56:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:35 smithi169 ceph-mon[33465]: osdmap e490: 8 total, 8 up, 8 in 2024-09-16T08:56:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-98", "mode": "writeback"} : dispatch 2024-09-16T08:56:35.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:56:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:35 smithi160 ceph-mon[30137]: osdmap e490: 8 total, 8 up, 8 in 2024-09-16T08:56:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-98", "mode": "writeback"} : dispatch 2024-09-16T08:56:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:56:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:35 smithi160 ceph-mon[40028]: osdmap e490: 8 total, 8 up, 8 in 2024-09-16T08:56:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-98", "mode": "writeback"} : dispatch 2024-09-16T08:56:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:36 smithi160 ceph-mon[30137]: pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-16T08:56:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:36 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:36 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:56:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-98", "mode": "writeback"}]': finished 2024-09-16T08:56:36.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:36 smithi160 ceph-mon[30137]: osdmap e491: 8 total, 8 up, 8 in 2024-09-16T08:56:36.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-98"} : dispatch 2024-09-16T08:56:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:36 smithi160 ceph-mon[40028]: pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-16T08:56:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:36 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:36 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:56:36.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-98", "mode": "writeback"}]': finished 2024-09-16T08:56:36.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:36 smithi160 ceph-mon[40028]: osdmap e491: 8 total, 8 up, 8 in 2024-09-16T08:56:36.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-98"} : dispatch 2024-09-16T08:56:36.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:36 smithi169 ceph-mon[33465]: pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-16T08:56:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:36 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:36 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:56:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-98", "mode": "writeback"}]': finished 2024-09-16T08:56:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:36 smithi169 ceph-mon[33465]: osdmap e491: 8 total, 8 up, 8 in 2024-09-16T08:56:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-98"} : dispatch 2024-09-16T08:56:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-98"}]': finished 2024-09-16T08:56:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:37 smithi160 ceph-mon[30137]: osdmap e492: 8 total, 8 up, 8 in 2024-09-16T08:56:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T08:56:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:37 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:56:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T08:56:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-98"}]': finished 2024-09-16T08:56:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:37 smithi160 ceph-mon[40028]: osdmap e492: 8 total, 8 up, 8 in 2024-09-16T08:56:37.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T08:56:37.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:37 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:56:37.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T08:56:37.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-98"}]': finished 2024-09-16T08:56:37.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:37 smithi169 ceph-mon[33465]: osdmap e492: 8 total, 8 up, 8 in 2024-09-16T08:56:37.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T08:56:37.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:37 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:56:37.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T08:56:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:38 smithi160 ceph-mon[30137]: pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T08:56:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:38 smithi160 ceph-mon[30137]: osdmap e493: 8 total, 8 up, 8 in 2024-09-16T08:56:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-16T08:56:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_count","val": "1"}]': finished 2024-09-16T08:56:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:38 smithi160 ceph-mon[30137]: osdmap e494: 8 total, 8 up, 8 in 2024-09-16T08:56:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:38 smithi160 ceph-mon[40028]: pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T08:56:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:38 smithi160 ceph-mon[40028]: osdmap e493: 8 total, 8 up, 8 in 2024-09-16T08:56:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-16T08:56:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_count","val": "1"}]': finished 2024-09-16T08:56:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:38 smithi160 ceph-mon[40028]: osdmap e494: 8 total, 8 up, 8 in 2024-09-16T08:56:38.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:38 smithi169 ceph-mon[33465]: pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T08:56:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:38 smithi169 ceph-mon[33465]: osdmap e493: 8 total, 8 up, 8 in 2024-09-16T08:56:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-16T08:56:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_count","val": "1"}]': finished 2024-09-16T08:56:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:38 smithi169 ceph-mon[33465]: osdmap e494: 8 total, 8 up, 8 in 2024-09-16T08:56:39.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:56:39.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:39 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:39.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:56:39.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:56:39.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:39 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:39.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:56:39.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:56:39.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:39 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:39.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:56:40.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:40 smithi160 ceph-mon[40028]: pgmap v751: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:40 smithi160 ceph-mon[40028]: osdmap e495: 8 total, 8 up, 8 in 2024-09-16T08:56:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-16T08:56:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "target_max_objects","val": "250"}]': finished 2024-09-16T08:56:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:40 smithi160 ceph-mon[40028]: osdmap e496: 8 total, 8 up, 8 in 2024-09-16T08:56:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:40 smithi160 ceph-mon[30137]: pgmap v751: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:40 smithi160 ceph-mon[30137]: osdmap e495: 8 total, 8 up, 8 in 2024-09-16T08:56:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-16T08:56:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "target_max_objects","val": "250"}]': finished 2024-09-16T08:56:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:40 smithi160 ceph-mon[30137]: osdmap e496: 8 total, 8 up, 8 in 2024-09-16T08:56:40.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:40 smithi169 ceph-mon[33465]: pgmap v751: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:40.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:40 smithi169 ceph-mon[33465]: osdmap e495: 8 total, 8 up, 8 in 2024-09-16T08:56:40.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-16T08:56:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-98","var": "target_max_objects","val": "250"}]': finished 2024-09-16T08:56:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:40 smithi169 ceph-mon[33465]: osdmap e496: 8 total, 8 up, 8 in 2024-09-16T08:56:41.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:56:41.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:56:41.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:56:42.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:42 smithi160 ceph-mon[40028]: pgmap v754: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:56:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:42 smithi160 ceph-mon[40028]: osdmap e497: 8 total, 8 up, 8 in 2024-09-16T08:56:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-98"} : dispatch 2024-09-16T08:56:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:42 smithi160 ceph-mon[30137]: pgmap v754: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:56:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:42 smithi160 ceph-mon[30137]: osdmap e497: 8 total, 8 up, 8 in 2024-09-16T08:56:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-98"} : dispatch 2024-09-16T08:56:42.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:42 smithi169 ceph-mon[33465]: pgmap v754: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:56:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:42 smithi169 ceph-mon[33465]: osdmap e497: 8 total, 8 up, 8 in 2024-09-16T08:56:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-98"} : dispatch 2024-09-16T08:56:42.850 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:56:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=sqlstore.transactions t=2024-09-16T08:56:42.505114602Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T08:56:42.850 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:56:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=cleanup t=2024-09-16T08:56:42.507804885Z level=info msg="Completed cleanup jobs" duration=13.344512ms 2024-09-16T08:56:42.850 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:56:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=sqlstore.transactions t=2024-09-16T08:56:42.515999055Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-16T08:56:42.850 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:56:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=grafana.update.checker t=2024-09-16T08:56:42.654728696Z level=info msg="Update check succeeded" duration=60.89804ms 2024-09-16T08:56:42.850 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 08:56:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=plugins.update.checker t=2024-09-16T08:56:42.656679355Z level=info msg="Update check succeeded" duration=66.3088ms 2024-09-16T08:56:43.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-98"}]': finished 2024-09-16T08:56:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:43 smithi169 ceph-mon[33465]: osdmap e498: 8 total, 8 up, 8 in 2024-09-16T08:56:43.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-98"}]': finished 2024-09-16T08:56:43.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:43 smithi160 ceph-mon[40028]: osdmap e498: 8 total, 8 up, 8 in 2024-09-16T08:56:43.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-98"}]': finished 2024-09-16T08:56:43.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:43 smithi160 ceph-mon[30137]: osdmap e498: 8 total, 8 up, 8 in 2024-09-16T08:56:44.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:44 smithi169 ceph-mon[33465]: pgmap v757: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:56:44.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:44 smithi169 ceph-mon[33465]: osdmap e499: 8 total, 8 up, 8 in 2024-09-16T08:56:44.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:44 smithi160 ceph-mon[40028]: pgmap v757: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:56:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:44 smithi160 ceph-mon[40028]: osdmap e499: 8 total, 8 up, 8 in 2024-09-16T08:56:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:44 smithi160 ceph-mon[30137]: pgmap v757: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:56:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:44 smithi160 ceph-mon[30137]: osdmap e499: 8 total, 8 up, 8 in 2024-09-16T08:56:45.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:56:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:56:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:56:45.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:45 smithi169 ceph-mon[33465]: osdmap e500: 8 total, 8 up, 8 in 2024-09-16T08:56:45.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:56:45.600 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:56:45 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:56:45.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:45 smithi160 ceph-mon[40028]: osdmap e500: 8 total, 8 up, 8 in 2024-09-16T08:56:45.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:56:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:45 smithi160 ceph-mon[30137]: osdmap e500: 8 total, 8 up, 8 in 2024-09-16T08:56:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:56:46.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:46 smithi169 ceph-mon[33465]: pgmap v760: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:46.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:46 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:46.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:46 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:46.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:56:46.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:46 smithi169 ceph-mon[33465]: osdmap e501: 8 total, 8 up, 8 in 2024-09-16T08:56:46.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:56:46.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:46 smithi160 ceph-mon[40028]: pgmap v760: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:46.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:46 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:46.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:46 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:46.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:56:46.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:46 smithi160 ceph-mon[40028]: osdmap e501: 8 total, 8 up, 8 in 2024-09-16T08:56:46.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:56:46.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:46 smithi160 ceph-mon[30137]: pgmap v760: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:46.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:46 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:46.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:46 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:46.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:56:46.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:46 smithi160 ceph-mon[30137]: osdmap e501: 8 total, 8 up, 8 in 2024-09-16T08:56:46.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:56:47.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:56:47.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:47 smithi169 ceph-mon[33465]: osdmap e502: 8 total, 8 up, 8 in 2024-09-16T08:56:47.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-100"} : dispatch 2024-09-16T08:56:47.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-100"}]': finished 2024-09-16T08:56:47.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:56:47.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:47 smithi160 ceph-mon[40028]: osdmap e502: 8 total, 8 up, 8 in 2024-09-16T08:56:47.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-100"} : dispatch 2024-09-16T08:56:47.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-100"}]': finished 2024-09-16T08:56:47.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:56:47.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:47 smithi160 ceph-mon[30137]: osdmap e502: 8 total, 8 up, 8 in 2024-09-16T08:56:47.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-100"} : dispatch 2024-09-16T08:56:47.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-6", "overlaypool": "test-rados-api-smithi160-76079-100"}]': finished 2024-09-16T08:56:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:48 smithi169 ceph-mon[33465]: pgmap v763: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:48 smithi169 ceph-mon[33465]: osdmap e503: 8 total, 8 up, 8 in 2024-09-16T08:56:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-100", "mode": "writeback"} : dispatch 2024-09-16T08:56:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:48 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:56:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-100", "mode": "writeback"}]': finished 2024-09-16T08:56:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:48 smithi169 ceph-mon[33465]: osdmap e504: 8 total, 8 up, 8 in 2024-09-16T08:56:48.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:48 smithi160 ceph-mon[30137]: pgmap v763: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:48.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:48 smithi160 ceph-mon[30137]: osdmap e503: 8 total, 8 up, 8 in 2024-09-16T08:56:48.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-100", "mode": "writeback"} : dispatch 2024-09-16T08:56:48.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:48 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:56:48.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-100", "mode": "writeback"}]': finished 2024-09-16T08:56:48.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:48 smithi160 ceph-mon[30137]: osdmap e504: 8 total, 8 up, 8 in 2024-09-16T08:56:48.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:48 smithi160 ceph-mon[40028]: pgmap v763: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:48.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:48 smithi160 ceph-mon[40028]: osdmap e503: 8 total, 8 up, 8 in 2024-09-16T08:56:48.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-100", "mode": "writeback"} : dispatch 2024-09-16T08:56:48.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:48 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:56:48.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-100", "mode": "writeback"}]': finished 2024-09-16T08:56:48.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:48 smithi160 ceph-mon[40028]: osdmap e504: 8 total, 8 up, 8 in 2024-09-16T08:56:49.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:56:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:56:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:56:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_count","val": "2"}]': finished 2024-09-16T08:56:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:49 smithi169 ceph-mon[33465]: osdmap e505: 8 total, 8 up, 8 in 2024-09-16T08:56:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:56:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:56:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:56:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_count","val": "2"}]': finished 2024-09-16T08:56:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:49 smithi160 ceph-mon[30137]: osdmap e505: 8 total, 8 up, 8 in 2024-09-16T08:56:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:56:49.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:56:49.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:56:49.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_count","val": "2"}]': finished 2024-09-16T08:56:49.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:49 smithi160 ceph-mon[40028]: osdmap e505: 8 total, 8 up, 8 in 2024-09-16T08:56:50.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:50 smithi169 ceph-mon[33465]: pgmap v766: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:50.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:56:50.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:56:50.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:50 smithi169 ceph-mon[33465]: osdmap e506: 8 total, 8 up, 8 in 2024-09-16T08:56:50.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-16T08:56:50.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:50 smithi160 ceph-mon[30137]: pgmap v766: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:50.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:56:50.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:56:50.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:50 smithi160 ceph-mon[30137]: osdmap e506: 8 total, 8 up, 8 in 2024-09-16T08:56:50.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-16T08:56:50.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:50 smithi160 ceph-mon[40028]: pgmap v766: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:50.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:56:50.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:56:50.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:50 smithi160 ceph-mon[40028]: osdmap e506: 8 total, 8 up, 8 in 2024-09-16T08:56:50.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-16T08:56:51.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:51 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:56:51.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-16T08:56:51.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:51 smithi169 ceph-mon[33465]: osdmap e507: 8 total, 8 up, 8 in 2024-09-16T08:56:51.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-16T08:56:51.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:51 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:56:51.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-16T08:56:51.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:51 smithi160 ceph-mon[30137]: osdmap e507: 8 total, 8 up, 8 in 2024-09-16T08:56:51.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-16T08:56:51.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:51 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:56:51.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-16T08:56:51.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:51 smithi160 ceph-mon[40028]: osdmap e507: 8 total, 8 up, 8 in 2024-09-16T08:56:51.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-16T08:56:52.852 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:52 smithi169 ceph-mon[33465]: pgmap v769: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:52.852 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-16T08:56:52.852 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:52 smithi169 ceph-mon[33465]: osdmap e508: 8 total, 8 up, 8 in 2024-09-16T08:56:52.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:52 smithi160 ceph-mon[30137]: pgmap v769: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:52.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-16T08:56:52.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:52 smithi160 ceph-mon[30137]: osdmap e508: 8 total, 8 up, 8 in 2024-09-16T08:56:52.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:52 smithi160 ceph-mon[40028]: pgmap v769: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:56:52.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-16T08:56:52.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:52 smithi160 ceph-mon[40028]: osdmap e508: 8 total, 8 up, 8 in 2024-09-16T08:56:53.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:56:53.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:56:53.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:56:54.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:54 smithi169 ceph-mon[33465]: pgmap v772: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:56:54.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:56:54.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:54 smithi169 ceph-mon[33465]: osdmap e509: 8 total, 8 up, 8 in 2024-09-16T08:56:54.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-100"} : dispatch 2024-09-16T08:56:54.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:54 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:54.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:54 smithi160 ceph-mon[30137]: pgmap v772: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:56:54.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:56:54.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:54 smithi160 ceph-mon[30137]: osdmap e509: 8 total, 8 up, 8 in 2024-09-16T08:56:54.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-100"} : dispatch 2024-09-16T08:56:54.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:54 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:54.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:54 smithi160 ceph-mon[40028]: pgmap v772: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:56:54.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"}]': finished 2024-09-16T08:56:54.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:54 smithi160 ceph-mon[40028]: osdmap e509: 8 total, 8 up, 8 in 2024-09-16T08:56:54.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-100"} : dispatch 2024-09-16T08:56:54.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:54 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:55.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:56:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:56:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:56:55.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:56:55 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:56:55.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-100"}]': finished 2024-09-16T08:56:55.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:55 smithi169 ceph-mon[33465]: osdmap e510: 8 total, 8 up, 8 in 2024-09-16T08:56:55.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-100"}]': finished 2024-09-16T08:56:55.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:55 smithi160 ceph-mon[40028]: osdmap e510: 8 total, 8 up, 8 in 2024-09-16T08:56:55.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-100"}]': finished 2024-09-16T08:56:55.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:55 smithi160 ceph-mon[30137]: osdmap e510: 8 total, 8 up, 8 in 2024-09-16T08:56:56.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:56 smithi169 ceph-mon[33465]: pgmap v775: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-16T08:56:56.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:56 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:56 smithi169 ceph-mon[33465]: osdmap e511: 8 total, 8 up, 8 in 2024-09-16T08:56:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:56 smithi160 ceph-mon[30137]: pgmap v775: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-16T08:56:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:56 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:56 smithi160 ceph-mon[30137]: osdmap e511: 8 total, 8 up, 8 in 2024-09-16T08:56:56.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:56 smithi160 ceph-mon[40028]: pgmap v775: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-16T08:56:56.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:56 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:56:56.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:56 smithi160 ceph-mon[40028]: osdmap e511: 8 total, 8 up, 8 in 2024-09-16T08:56:57.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:57 smithi169 ceph-mon[33465]: osdmap e512: 8 total, 8 up, 8 in 2024-09-16T08:56:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:57 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:56:57.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:57 smithi160 ceph-mon[30137]: osdmap e512: 8 total, 8 up, 8 in 2024-09-16T08:56:57.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:56:57.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:57 smithi160 ceph-mon[40028]: osdmap e512: 8 total, 8 up, 8 in 2024-09-16T08:56:57.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:57 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:56:58.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:58 smithi169 ceph-mon[33465]: pgmap v778: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:56:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:58 smithi169 ceph-mon[33465]: osdmap e513: 8 total, 8 up, 8 in 2024-09-16T08:56:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:56:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:56:58.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:58 smithi160 ceph-mon[30137]: pgmap v778: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:58.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:56:58.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:58 smithi160 ceph-mon[30137]: osdmap e513: 8 total, 8 up, 8 in 2024-09-16T08:56:58.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:56:58.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:56:58.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:58 smithi160 ceph-mon[40028]: pgmap v778: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:56:58.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:56:58.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:58 smithi160 ceph-mon[40028]: osdmap e513: 8 total, 8 up, 8 in 2024-09-16T08:56:58.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-16T08:56:58.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-16T08:56:59.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:56:59.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:59 smithi169 ceph-mon[33465]: osdmap e514: 8 total, 8 up, 8 in 2024-09-16T08:56:59.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-16T08:56:59.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:59 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:59.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-16T08:56:59.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:59 smithi169 ceph-mon[33465]: osdmap e515: 8 total, 8 up, 8 in 2024-09-16T08:56:59.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:56:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:56:59.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:56:59.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:59 smithi160 ceph-mon[30137]: osdmap e514: 8 total, 8 up, 8 in 2024-09-16T08:56:59.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-16T08:56:59.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:59 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:59.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-16T08:56:59.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:59 smithi160 ceph-mon[30137]: osdmap e515: 8 total, 8 up, 8 in 2024-09-16T08:56:59.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:56:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:56:59.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-16T08:56:59.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:59 smithi160 ceph-mon[40028]: osdmap e514: 8 total, 8 up, 8 in 2024-09-16T08:56:59.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-16T08:56:59.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:59 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:56:59.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-16T08:56:59.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:59 smithi160 ceph-mon[40028]: osdmap e515: 8 total, 8 up, 8 in 2024-09-16T08:56:59.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:56:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-16T08:57:00.849 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:00 smithi160 ceph-mon[40028]: pgmap v781: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:00.849 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:57:00.849 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:00 smithi160 ceph-mon[40028]: osdmap e516: 8 total, 8 up, 8 in 2024-09-16T08:57:00.850 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:00 smithi160 ceph-mon[30137]: pgmap v781: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:00.850 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:57:00.850 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:00 smithi160 ceph-mon[30137]: osdmap e516: 8 total, 8 up, 8 in 2024-09-16T08:57:00.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:00 smithi169 ceph-mon[33465]: pgmap v781: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:00.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-16T08:57:00.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:00 smithi169 ceph-mon[33465]: osdmap e516: 8 total, 8 up, 8 in 2024-09-16T08:57:01.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:57:01.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-102"} : dispatch 2024-09-16T08:57:01.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:57:01.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-102"} : dispatch 2024-09-16T08:57:01.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-6"} : dispatch 2024-09-16T08:57:01.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3302936851' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-6", "tierpool": "test-rados-api-smithi160-76079-102"} : dispatch 2024-09-16T08:57:02.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:02 smithi169 ceph-mon[33465]: pgmap v784: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:02.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:02 smithi169 ceph-mon[33465]: osdmap e517: 8 total, 8 up, 8 in 2024-09-16T08:57:02.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:02 smithi160 ceph-mon[40028]: pgmap v784: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:02.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:02 smithi160 ceph-mon[40028]: osdmap e517: 8 total, 8 up, 8 in 2024-09-16T08:57:02.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:02 smithi160 ceph-mon[30137]: pgmap v784: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:02.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:02 smithi160 ceph-mon[30137]: osdmap e517: 8 total, 8 up, 8 in 2024-09-16T08:57:03.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:03 smithi169 ceph-mon[33465]: osdmap e518: 8 total, 8 up, 8 in 2024-09-16T08:57:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:03 smithi169 ceph-mon[33465]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:03 smithi169 ceph-mon[33465]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:57:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:03 smithi169 ceph-mon[33465]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:57:03.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:03 smithi160 ceph-mon[30137]: osdmap e518: 8 total, 8 up, 8 in 2024-09-16T08:57:03.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:03.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:03 smithi160 ceph-mon[30137]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:03.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:03.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:03 smithi160 ceph-mon[30137]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:03.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:57:03.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:03 smithi160 ceph-mon[30137]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:57:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:03 smithi160 ceph-mon[40028]: osdmap e518: 8 total, 8 up, 8 in 2024-09-16T08:57:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:03 smithi160 ceph-mon[40028]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:03 smithi160 ceph-mon[40028]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:57:03.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:03 smithi160 ceph-mon[40028]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:57:04.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:04 smithi169 ceph-mon[33465]: pgmap v787: 164 pgs: 164 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:04 smithi169 ceph-mon[33465]: from='client.57245 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:57:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:04 smithi169 ceph-mon[33465]: osdmap e519: 8 total, 8 up, 8 in 2024-09-16T08:57:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi160-76079-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:04 smithi169 ceph-mon[33465]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi160-76079-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:57:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:57:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:04 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-mon[30137]: pgmap v787: 164 pgs: 164 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-mon[30137]: from='client.57245 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:57:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-mon[30137]: osdmap e519: 8 total, 8 up, 8 in 2024-09-16T08:57:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi160-76079-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-mon[30137]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi160-76079-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:57:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:57:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:04.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-mon[40028]: pgmap v787: 164 pgs: 164 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:04.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-mon[40028]: from='client.57245 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:57:04.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-mon[40028]: osdmap e519: 8 total, 8 up, 8 in 2024-09-16T08:57:04.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi160-76079-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:04.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-mon[40028]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi160-76079-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:04.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:57:04.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:57:04.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:05.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:57:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:57:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:57:05.597 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:57:05 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:57:05.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:05 smithi169 ceph-mon[33465]: osdmap e520: 8 total, 8 up, 8 in 2024-09-16T08:57:05.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:05 smithi169 ceph-mon[33465]: from='client.57245 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi160-76079-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi160-76079-104"}]': finished 2024-09-16T08:57:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:05 smithi169 ceph-mon[33465]: osdmap e521: 8 total, 8 up, 8 in 2024-09-16T08:57:05.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:05 smithi160 ceph-mon[30137]: osdmap e520: 8 total, 8 up, 8 in 2024-09-16T08:57:05.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:05 smithi160 ceph-mon[30137]: from='client.57245 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi160-76079-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi160-76079-104"}]': finished 2024-09-16T08:57:05.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:05 smithi160 ceph-mon[30137]: osdmap e521: 8 total, 8 up, 8 in 2024-09-16T08:57:05.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:05 smithi160 ceph-mon[40028]: osdmap e520: 8 total, 8 up, 8 in 2024-09-16T08:57:05.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:05 smithi160 ceph-mon[40028]: from='client.57245 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi160-76079-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi160-76079-104"}]': finished 2024-09-16T08:57:05.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:05 smithi160 ceph-mon[40028]: osdmap e521: 8 total, 8 up, 8 in 2024-09-16T08:57:06.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:06 smithi169 ceph-mon[33465]: pgmap v790: 164 pgs: 164 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:06 smithi169 ceph-mon[33465]: osdmap e522: 8 total, 8 up, 8 in 2024-09-16T08:57:06.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:06 smithi160 ceph-mon[40028]: pgmap v790: 164 pgs: 164 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:06.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:06.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:06 smithi160 ceph-mon[40028]: osdmap e522: 8 total, 8 up, 8 in 2024-09-16T08:57:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:06 smithi160 ceph-mon[30137]: pgmap v790: 164 pgs: 164 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:06 smithi160 ceph-mon[30137]: osdmap e522: 8 total, 8 up, 8 in 2024-09-16T08:57:08.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:08 smithi169 ceph-mon[33465]: pgmap v793: 172 pgs: 8 unknown, 164 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:08 smithi169 ceph-mon[33465]: osdmap e523: 8 total, 8 up, 8 in 2024-09-16T08:57:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:08 smithi169 ceph-mon[33465]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:08 smithi160 ceph-mon[30137]: pgmap v793: 172 pgs: 8 unknown, 164 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:08 smithi160 ceph-mon[30137]: osdmap e523: 8 total, 8 up, 8 in 2024-09-16T08:57:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:08.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:08 smithi160 ceph-mon[30137]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:08.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:08 smithi160 ceph-mon[40028]: pgmap v793: 172 pgs: 8 unknown, 164 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:08.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:08 smithi160 ceph-mon[40028]: osdmap e523: 8 total, 8 up, 8 in 2024-09-16T08:57:08.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:08.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:08 smithi160 ceph-mon[40028]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:09.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:09 smithi160 ceph-mon[30137]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:57:09.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:09 smithi160 ceph-mon[30137]: osdmap e524: 8 total, 8 up, 8 in 2024-09-16T08:57:09.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:09 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:09.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:09 smithi160 ceph-mon[40028]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:57:09.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:09 smithi160 ceph-mon[40028]: osdmap e524: 8 total, 8 up, 8 in 2024-09-16T08:57:09.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:09 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:10.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:09 smithi169 ceph-mon[33465]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:57:10.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:09 smithi169 ceph-mon[33465]: osdmap e524: 8 total, 8 up, 8 in 2024-09-16T08:57:10.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:09 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:10.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:10 smithi160 ceph-mon[30137]: pgmap v796: 204 pgs: 204 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:57:10.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:10 smithi160 ceph-mon[30137]: osdmap e525: 8 total, 8 up, 8 in 2024-09-16T08:57:10.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:10.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:10 smithi160 ceph-mon[30137]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:10.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:10 smithi160 ceph-mon[40028]: pgmap v796: 204 pgs: 204 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:57:10.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:10 smithi160 ceph-mon[40028]: osdmap e525: 8 total, 8 up, 8 in 2024-09-16T08:57:10.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:10.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:10 smithi160 ceph-mon[40028]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:11.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:10 smithi169 ceph-mon[33465]: pgmap v796: 204 pgs: 204 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:57:11.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:10 smithi169 ceph-mon[33465]: osdmap e525: 8 total, 8 up, 8 in 2024-09-16T08:57:11.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:11.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:10 smithi169 ceph-mon[33465]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:11 smithi160 ceph-mon[30137]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:57:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:11 smithi160 ceph-mon[30137]: osdmap e526: 8 total, 8 up, 8 in 2024-09-16T08:57:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:11 smithi160 ceph-mon[30137]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:11.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:11 smithi160 ceph-mon[40028]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:57:11.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:11 smithi160 ceph-mon[40028]: osdmap e526: 8 total, 8 up, 8 in 2024-09-16T08:57:11.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:11.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:11 smithi160 ceph-mon[40028]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:12.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:11 smithi169 ceph-mon[33465]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:57:12.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:11 smithi169 ceph-mon[33465]: osdmap e526: 8 total, 8 up, 8 in 2024-09-16T08:57:12.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:12.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:11 smithi169 ceph-mon[33465]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:12.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:12 smithi160 ceph-mon[30137]: pgmap v799: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:57:12.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:12 smithi160 ceph-mon[30137]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"}]': finished 2024-09-16T08:57:12.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:12 smithi160 ceph-mon[30137]: osdmap e527: 8 total, 8 up, 8 in 2024-09-16T08:57:12.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:12 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-107", "overlaypool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:12.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:12 smithi160 ceph-mon[30137]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-107", "overlaypool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:12.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:12 smithi160 ceph-mon[40028]: pgmap v799: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:57:12.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:12 smithi160 ceph-mon[40028]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"}]': finished 2024-09-16T08:57:12.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:12 smithi160 ceph-mon[40028]: osdmap e527: 8 total, 8 up, 8 in 2024-09-16T08:57:12.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:12 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-107", "overlaypool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:12.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:12 smithi160 ceph-mon[40028]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-107", "overlaypool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:13.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:12 smithi169 ceph-mon[33465]: pgmap v799: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:57:13.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:12 smithi169 ceph-mon[33465]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"}]': finished 2024-09-16T08:57:13.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:12 smithi169 ceph-mon[33465]: osdmap e527: 8 total, 8 up, 8 in 2024-09-16T08:57:13.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:12 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-107", "overlaypool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:13.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:12 smithi169 ceph-mon[33465]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-107", "overlaypool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:13.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:13 smithi160 ceph-mon[30137]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-107", "overlaypool": "test-rados-api-smithi160-76079-107-cache"}]': finished 2024-09-16T08:57:13.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:13 smithi160 ceph-mon[30137]: osdmap e528: 8 total, 8 up, 8 in 2024-09-16T08:57:13.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-107-cache", "mode": "writeback"} : dispatch 2024-09-16T08:57:13.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:13 smithi160 ceph-mon[30137]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-107-cache", "mode": "writeback"} : dispatch 2024-09-16T08:57:13.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:13 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:57:13.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:13 smithi160 ceph-mon[30137]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-107-cache", "mode": "writeback"}]': finished 2024-09-16T08:57:13.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:13 smithi160 ceph-mon[30137]: osdmap e529: 8 total, 8 up, 8 in 2024-09-16T08:57:13.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:13 smithi160 ceph-mon[40028]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-107", "overlaypool": "test-rados-api-smithi160-76079-107-cache"}]': finished 2024-09-16T08:57:13.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:13 smithi160 ceph-mon[40028]: osdmap e528: 8 total, 8 up, 8 in 2024-09-16T08:57:13.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-107-cache", "mode": "writeback"} : dispatch 2024-09-16T08:57:13.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:13 smithi160 ceph-mon[40028]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-107-cache", "mode": "writeback"} : dispatch 2024-09-16T08:57:13.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:13 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:57:13.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:13 smithi160 ceph-mon[40028]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-107-cache", "mode": "writeback"}]': finished 2024-09-16T08:57:13.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:13 smithi160 ceph-mon[40028]: osdmap e529: 8 total, 8 up, 8 in 2024-09-16T08:57:14.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:13 smithi169 ceph-mon[33465]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-107", "overlaypool": "test-rados-api-smithi160-76079-107-cache"}]': finished 2024-09-16T08:57:14.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:13 smithi169 ceph-mon[33465]: osdmap e528: 8 total, 8 up, 8 in 2024-09-16T08:57:14.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-107-cache", "mode": "writeback"} : dispatch 2024-09-16T08:57:14.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:13 smithi169 ceph-mon[33465]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-107-cache", "mode": "writeback"} : dispatch 2024-09-16T08:57:14.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:13 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:57:14.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:13 smithi169 ceph-mon[33465]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-107-cache", "mode": "writeback"}]': finished 2024-09-16T08:57:14.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:13 smithi169 ceph-mon[33465]: osdmap e529: 8 total, 8 up, 8 in 2024-09-16T08:57:14.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:14 smithi160 ceph-mon[30137]: pgmap v802: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:14.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-107"} : dispatch 2024-09-16T08:57:14.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:14 smithi160 ceph-mon[30137]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-107"} : dispatch 2024-09-16T08:57:14.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:14 smithi160 ceph-mon[40028]: pgmap v802: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:14.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:14 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-107"} : dispatch 2024-09-16T08:57:14.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:14 smithi160 ceph-mon[40028]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-107"} : dispatch 2024-09-16T08:57:15.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:14 smithi169 ceph-mon[33465]: pgmap v802: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:15.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:14 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-107"} : dispatch 2024-09-16T08:57:15.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:14 smithi169 ceph-mon[33465]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-107"} : dispatch 2024-09-16T08:57:15.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:57:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:57:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:57:15.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:57:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:57:15.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:15 smithi160 ceph-mon[30137]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-107"}]': finished 2024-09-16T08:57:15.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:15 smithi160 ceph-mon[30137]: osdmap e530: 8 total, 8 up, 8 in 2024-09-16T08:57:15.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:15 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:15.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:15 smithi160 ceph-mon[30137]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:15.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:15 smithi160 ceph-mon[40028]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-107"}]': finished 2024-09-16T08:57:15.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:15 smithi160 ceph-mon[40028]: osdmap e530: 8 total, 8 up, 8 in 2024-09-16T08:57:15.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:15 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:15.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:15 smithi160 ceph-mon[40028]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:16.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:15 smithi169 ceph-mon[33465]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-107"}]': finished 2024-09-16T08:57:16.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:15 smithi169 ceph-mon[33465]: osdmap e530: 8 total, 8 up, 8 in 2024-09-16T08:57:16.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:15 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2158891641' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:16.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:15 smithi169 ceph-mon[33465]: from='client.57251 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"} : dispatch 2024-09-16T08:57:16.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:16 smithi160 ceph-mon[30137]: pgmap v805: 236 pgs: 236 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:57:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:16 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:57:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:16 smithi160 ceph-mon[30137]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"}]': finished 2024-09-16T08:57:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:16 smithi160 ceph-mon[30137]: osdmap e531: 8 total, 8 up, 8 in 2024-09-16T08:57:16.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:16 smithi160 ceph-mon[40028]: pgmap v805: 236 pgs: 236 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:57:16.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:16.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:16 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:57:16.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:16 smithi160 ceph-mon[40028]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"}]': finished 2024-09-16T08:57:16.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:16 smithi160 ceph-mon[40028]: osdmap e531: 8 total, 8 up, 8 in 2024-09-16T08:57:17.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:16 smithi169 ceph-mon[33465]: pgmap v805: 236 pgs: 236 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:57:17.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:17.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:16 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:57:17.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:16 smithi169 ceph-mon[33465]: from='client.57251 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-107", "tierpool": "test-rados-api-smithi160-76079-107-cache"}]': finished 2024-09-16T08:57:17.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:16 smithi169 ceph-mon[33465]: osdmap e531: 8 total, 8 up, 8 in 2024-09-16T08:57:17.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:17 smithi160 ceph-mon[30137]: osdmap e532: 8 total, 8 up, 8 in 2024-09-16T08:57:17.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:17 smithi160 ceph-mon[40028]: osdmap e532: 8 total, 8 up, 8 in 2024-09-16T08:57:18.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:17 smithi169 ceph-mon[33465]: osdmap e532: 8 total, 8 up, 8 in 2024-09-16T08:57:18.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:18 smithi160 ceph-mon[30137]: pgmap v808: 204 pgs: 204 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:57:18.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:18 smithi160 ceph-mon[30137]: osdmap e533: 8 total, 8 up, 8 in 2024-09-16T08:57:18.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:18.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:18.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-76079-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:57:18.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:18 smithi160 ceph-mon[40028]: pgmap v808: 204 pgs: 204 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:57:18.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:18 smithi160 ceph-mon[40028]: osdmap e533: 8 total, 8 up, 8 in 2024-09-16T08:57:18.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:18.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:18.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-76079-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:57:19.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:18 smithi169 ceph-mon[33465]: pgmap v808: 204 pgs: 204 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:57:19.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:18 smithi169 ceph-mon[33465]: osdmap e533: 8 total, 8 up, 8 in 2024-09-16T08:57:19.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:19.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:19.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-76079-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:57:19.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-76079-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:57:19.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:19 smithi160 ceph-mon[30137]: osdmap e534: 8 total, 8 up, 8 in 2024-09-16T08:57:19.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76079-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:19.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:57:19.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-76079-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:57:19.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:19 smithi160 ceph-mon[40028]: osdmap e534: 8 total, 8 up, 8 in 2024-09-16T08:57:19.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76079-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:19.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:57:20.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-76079-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:57:20.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:19 smithi169 ceph-mon[33465]: osdmap e534: 8 total, 8 up, 8 in 2024-09-16T08:57:20.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76079-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:20.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:57:20.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:20 smithi160 ceph-mon[30137]: pgmap v811: 172 pgs: 172 active+clean; 583 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:20.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:20 smithi160 ceph-mon[30137]: osdmap e535: 8 total, 8 up, 8 in 2024-09-16T08:57:20.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:20 smithi160 ceph-mon[40028]: pgmap v811: 172 pgs: 172 active+clean; 583 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:20.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:20 smithi160 ceph-mon[40028]: osdmap e535: 8 total, 8 up, 8 in 2024-09-16T08:57:21.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:20 smithi169 ceph-mon[33465]: pgmap v811: 172 pgs: 172 active+clean; 583 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:21.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:20 smithi169 ceph-mon[33465]: osdmap e535: 8 total, 8 up, 8 in 2024-09-16T08:57:22.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:21 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76079-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-76079-109"}]': finished 2024-09-16T08:57:22.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:21 smithi169 ceph-mon[33465]: osdmap e536: 8 total, 8 up, 8 in 2024-09-16T08:57:22.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:21 smithi169 ceph-mon[33465]: pgmap v814: 180 pgs: 8 unknown, 172 active+clean; 583 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:22.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:21 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76079-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-76079-109"}]': finished 2024-09-16T08:57:22.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:21 smithi160 ceph-mon[30137]: osdmap e536: 8 total, 8 up, 8 in 2024-09-16T08:57:22.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:21 smithi160 ceph-mon[30137]: pgmap v814: 180 pgs: 8 unknown, 172 active+clean; 583 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:22.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:21 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76079-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-76079-109"}]': finished 2024-09-16T08:57:22.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:21 smithi160 ceph-mon[40028]: osdmap e536: 8 total, 8 up, 8 in 2024-09-16T08:57:22.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:21 smithi160 ceph-mon[40028]: pgmap v814: 180 pgs: 8 unknown, 172 active+clean; 583 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:23.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:22 smithi169 ceph-mon[33465]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:23.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:22 smithi169 ceph-mon[33465]: osdmap e537: 8 total, 8 up, 8 in 2024-09-16T08:57:23.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:22 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-109-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:23.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:22 smithi160 ceph-mon[30137]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:23.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:22 smithi160 ceph-mon[30137]: osdmap e537: 8 total, 8 up, 8 in 2024-09-16T08:57:23.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:22 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-109-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:23.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:22 smithi160 ceph-mon[40028]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:23.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:22 smithi160 ceph-mon[40028]: osdmap e537: 8 total, 8 up, 8 in 2024-09-16T08:57:23.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:22 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-109-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:24.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:23 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:57:24.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:23 smithi169 ceph-mon[33465]: osdmap e538: 8 total, 8 up, 8 in 2024-09-16T08:57:24.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:23 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-109", "tierpool": "test-rados-api-smithi160-76079-109-cache"} : dispatch 2024-09-16T08:57:24.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:23 smithi169 ceph-mon[33465]: pgmap v817: 212 pgs: 40 unknown, 172 active+clean; 583 KiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:24.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:23 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:57:24.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:23 smithi160 ceph-mon[30137]: osdmap e538: 8 total, 8 up, 8 in 2024-09-16T08:57:24.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:23 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-109", "tierpool": "test-rados-api-smithi160-76079-109-cache"} : dispatch 2024-09-16T08:57:24.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:23 smithi160 ceph-mon[30137]: pgmap v817: 212 pgs: 40 unknown, 172 active+clean; 583 KiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:24.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:23 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:57:24.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:23 smithi160 ceph-mon[40028]: osdmap e538: 8 total, 8 up, 8 in 2024-09-16T08:57:24.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:23 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-109", "tierpool": "test-rados-api-smithi160-76079-109-cache"} : dispatch 2024-09-16T08:57:24.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:23 smithi160 ceph-mon[40028]: pgmap v817: 212 pgs: 40 unknown, 172 active+clean; 583 KiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:24.988 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-109", "tierpool": "test-rados-api-smithi160-76079-109-cache"}]': finished 2024-09-16T08:57:24.988 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:24 smithi160 ceph-mon[30137]: osdmap e539: 8 total, 8 up, 8 in 2024-09-16T08:57:24.989 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-109", "overlaypool": "test-rados-api-smithi160-76079-109-cache"} : dispatch 2024-09-16T08:57:24.989 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:24 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:57:24.989 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-109", "tierpool": "test-rados-api-smithi160-76079-109-cache"}]': finished 2024-09-16T08:57:24.989 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:24 smithi160 ceph-mon[40028]: osdmap e539: 8 total, 8 up, 8 in 2024-09-16T08:57:24.989 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-109", "overlaypool": "test-rados-api-smithi160-76079-109-cache"} : dispatch 2024-09-16T08:57:24.989 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:24 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:57:25.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-109", "tierpool": "test-rados-api-smithi160-76079-109-cache"}]': finished 2024-09-16T08:57:25.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:24 smithi169 ceph-mon[33465]: osdmap e539: 8 total, 8 up, 8 in 2024-09-16T08:57:25.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-109", "overlaypool": "test-rados-api-smithi160-76079-109-cache"} : dispatch 2024-09-16T08:57:25.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:24 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:57:25.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:57:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:57:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:57:25.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:57:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:57:26.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-109", "overlaypool": "test-rados-api-smithi160-76079-109-cache"}]': finished 2024-09-16T08:57:26.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:25 smithi169 ceph-mon[33465]: osdmap e540: 8 total, 8 up, 8 in 2024-09-16T08:57:26.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-109-cache", "mode": "writeback"} : dispatch 2024-09-16T08:57:26.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:25 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:57:26.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:25 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:57:26.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:25 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:57:26.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:25 smithi169 ceph-mon[33465]: pgmap v820: 212 pgs: 212 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:26.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:25 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:26.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:25 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:57:26.101 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-109-cache", "mode": "writeback"}]': finished 2024-09-16T08:57:26.101 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:25 smithi169 ceph-mon[33465]: osdmap e541: 8 total, 8 up, 8 in 2024-09-16T08:57:26.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-109", "overlaypool": "test-rados-api-smithi160-76079-109-cache"}]': finished 2024-09-16T08:57:26.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[40028]: osdmap e540: 8 total, 8 up, 8 in 2024-09-16T08:57:26.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-109-cache", "mode": "writeback"} : dispatch 2024-09-16T08:57:26.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:57:26.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:57:26.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:57:26.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[40028]: pgmap v820: 212 pgs: 212 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:26.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:26.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:57:26.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-109-cache", "mode": "writeback"}]': finished 2024-09-16T08:57:26.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[40028]: osdmap e541: 8 total, 8 up, 8 in 2024-09-16T08:57:26.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-109", "overlaypool": "test-rados-api-smithi160-76079-109-cache"}]': finished 2024-09-16T08:57:26.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[30137]: osdmap e540: 8 total, 8 up, 8 in 2024-09-16T08:57:26.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-109-cache", "mode": "writeback"} : dispatch 2024-09-16T08:57:26.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:57:26.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:57:26.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:57:26.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[30137]: pgmap v820: 212 pgs: 212 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:26.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:26.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:57:26.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-109-cache", "mode": "writeback"}]': finished 2024-09-16T08:57:26.199 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:25 smithi160 ceph-mon[30137]: osdmap e541: 8 total, 8 up, 8 in 2024-09-16T08:57:27.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:57:27.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-16T08:57:27.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:57:27.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-16T08:57:27.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:57:27.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-16T08:57:28.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:27 smithi169 ceph-mon[33465]: osdmap e542: 8 total, 8 up, 8 in 2024-09-16T08:57:28.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:57:28.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:27 smithi169 ceph-mon[33465]: pgmap v823: 212 pgs: 212 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:28.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:57:28.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:27 smithi160 ceph-mon[30137]: osdmap e542: 8 total, 8 up, 8 in 2024-09-16T08:57:28.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:57:28.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:27 smithi160 ceph-mon[30137]: pgmap v823: 212 pgs: 212 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:28.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:57:28.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:27 smithi160 ceph-mon[40028]: osdmap e542: 8 total, 8 up, 8 in 2024-09-16T08:57:28.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T08:57:28.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:27 smithi160 ceph-mon[40028]: pgmap v823: 212 pgs: 212 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:28.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-16T08:57:29.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:28 smithi169 ceph-mon[33465]: osdmap e543: 8 total, 8 up, 8 in 2024-09-16T08:57:29.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-16T08:57:29.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:28 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:57:29.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-16T08:57:29.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:28 smithi169 ceph-mon[33465]: osdmap e544: 8 total, 8 up, 8 in 2024-09-16T08:57:29.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "min_read_recency_for_promote","val": "4"} : dispatch 2024-09-16T08:57:29.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:28 smithi160 ceph-mon[30137]: osdmap e543: 8 total, 8 up, 8 in 2024-09-16T08:57:29.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-16T08:57:29.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:28 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:57:29.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-16T08:57:29.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:28 smithi160 ceph-mon[30137]: osdmap e544: 8 total, 8 up, 8 in 2024-09-16T08:57:29.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "min_read_recency_for_promote","val": "4"} : dispatch 2024-09-16T08:57:29.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:28 smithi160 ceph-mon[40028]: osdmap e543: 8 total, 8 up, 8 in 2024-09-16T08:57:29.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-16T08:57:29.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:28 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:57:29.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-16T08:57:29.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:28 smithi160 ceph-mon[40028]: osdmap e544: 8 total, 8 up, 8 in 2024-09-16T08:57:29.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "min_read_recency_for_promote","val": "4"} : dispatch 2024-09-16T08:57:30.099 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:29 smithi160 ceph-mon[30137]: pgmap v826: 212 pgs: 212 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:30.099 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-16T08:57:30.099 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:29 smithi160 ceph-mon[30137]: osdmap e545: 8 total, 8 up, 8 in 2024-09-16T08:57:30.099 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:29 smithi160 ceph-mon[40028]: pgmap v826: 212 pgs: 212 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:30.099 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-16T08:57:30.100 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:29 smithi160 ceph-mon[40028]: osdmap e545: 8 total, 8 up, 8 in 2024-09-16T08:57:30.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:29 smithi169 ceph-mon[33465]: pgmap v826: 212 pgs: 212 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:30.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-16T08:57:30.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:29 smithi169 ceph-mon[33465]: osdmap e545: 8 total, 8 up, 8 in 2024-09-16T08:57:31.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:31.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:31.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:32.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-109"}]': finished 2024-09-16T08:57:32.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:31 smithi169 ceph-mon[33465]: osdmap e546: 8 total, 8 up, 8 in 2024-09-16T08:57:32.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-109", "tierpool": "test-rados-api-smithi160-76079-109-cache"} : dispatch 2024-09-16T08:57:32.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:31 smithi169 ceph-mon[33465]: pgmap v829: 212 pgs: 212 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:32.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-109"}]': finished 2024-09-16T08:57:32.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:31 smithi160 ceph-mon[30137]: osdmap e546: 8 total, 8 up, 8 in 2024-09-16T08:57:32.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-109", "tierpool": "test-rados-api-smithi160-76079-109-cache"} : dispatch 2024-09-16T08:57:32.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:31 smithi160 ceph-mon[30137]: pgmap v829: 212 pgs: 212 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:32.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-109"}]': finished 2024-09-16T08:57:32.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:31 smithi160 ceph-mon[40028]: osdmap e546: 8 total, 8 up, 8 in 2024-09-16T08:57:32.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-109", "tierpool": "test-rados-api-smithi160-76079-109-cache"} : dispatch 2024-09-16T08:57:32.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:31 smithi160 ceph-mon[40028]: pgmap v829: 212 pgs: 212 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:33.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-109", "tierpool": "test-rados-api-smithi160-76079-109-cache"}]': finished 2024-09-16T08:57:33.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:32 smithi169 ceph-mon[33465]: osdmap e547: 8 total, 8 up, 8 in 2024-09-16T08:57:33.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-109", "tierpool": "test-rados-api-smithi160-76079-109-cache"}]': finished 2024-09-16T08:57:33.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:32 smithi160 ceph-mon[30137]: osdmap e547: 8 total, 8 up, 8 in 2024-09-16T08:57:33.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-109", "tierpool": "test-rados-api-smithi160-76079-109-cache"}]': finished 2024-09-16T08:57:33.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:32 smithi160 ceph-mon[40028]: osdmap e547: 8 total, 8 up, 8 in 2024-09-16T08:57:34.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:33 smithi169 ceph-mon[33465]: osdmap e548: 8 total, 8 up, 8 in 2024-09-16T08:57:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:33 smithi169 ceph-mon[33465]: pgmap v832: 180 pgs: 180 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:33 smithi169 ceph-mon[33465]: osdmap e549: 8 total, 8 up, 8 in 2024-09-16T08:57:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:34.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:33 smithi160 ceph-mon[30137]: osdmap e548: 8 total, 8 up, 8 in 2024-09-16T08:57:34.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:33 smithi160 ceph-mon[30137]: pgmap v832: 180 pgs: 180 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:34.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:33 smithi160 ceph-mon[30137]: osdmap e549: 8 total, 8 up, 8 in 2024-09-16T08:57:34.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:33 smithi160 ceph-mon[40028]: osdmap e548: 8 total, 8 up, 8 in 2024-09-16T08:57:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:33 smithi160 ceph-mon[40028]: pgmap v832: 180 pgs: 180 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:33 smithi160 ceph-mon[40028]: osdmap e549: 8 total, 8 up, 8 in 2024-09-16T08:57:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:35.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:57:35.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-109"}]': finished 2024-09-16T08:57:35.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:34 smithi169 ceph-mon[33465]: osdmap e550: 8 total, 8 up, 8 in 2024-09-16T08:57:35.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:35.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:57:35.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-109"}]': finished 2024-09-16T08:57:35.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:34 smithi160 ceph-mon[30137]: osdmap e550: 8 total, 8 up, 8 in 2024-09-16T08:57:35.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:35.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:57:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:57:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:57:35.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:57:35.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-109"}]': finished 2024-09-16T08:57:35.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:34 smithi160 ceph-mon[40028]: osdmap e550: 8 total, 8 up, 8 in 2024-09-16T08:57:35.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-109"} : dispatch 2024-09-16T08:57:35.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:57:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:57:36.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:35 smithi169 ceph-mon[33465]: pgmap v835: 172 pgs: 172 active+clean; 583 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:36.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:35 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:36.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-109"}]': finished 2024-09-16T08:57:36.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:35 smithi169 ceph-mon[33465]: osdmap e551: 8 total, 8 up, 8 in 2024-09-16T08:57:36.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:35 smithi160 ceph-mon[30137]: pgmap v835: 172 pgs: 172 active+clean; 583 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:36.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:35 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:36.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-109"}]': finished 2024-09-16T08:57:36.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:35 smithi160 ceph-mon[30137]: osdmap e551: 8 total, 8 up, 8 in 2024-09-16T08:57:36.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:35 smithi160 ceph-mon[40028]: pgmap v835: 172 pgs: 172 active+clean; 583 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:36.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:35 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:36.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3864370940' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-109"}]': finished 2024-09-16T08:57:36.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:35 smithi160 ceph-mon[40028]: osdmap e551: 8 total, 8 up, 8 in 2024-09-16T08:57:37.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:36 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:37.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:36 smithi169 ceph-mon[33465]: osdmap e552: 8 total, 8 up, 8 in 2024-09-16T08:57:37.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:37.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:36 smithi169 ceph-mon[33465]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:37.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:36 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:37.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:36 smithi160 ceph-mon[30137]: osdmap e552: 8 total, 8 up, 8 in 2024-09-16T08:57:37.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:37.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:36 smithi160 ceph-mon[30137]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:37.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:36 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:37.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:36 smithi160 ceph-mon[40028]: osdmap e552: 8 total, 8 up, 8 in 2024-09-16T08:57:37.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:37.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:36 smithi160 ceph-mon[40028]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:38.101 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:37 smithi169 ceph-mon[33465]: pgmap v838: 164 pgs: 164 active+clean; 583 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-16T08:57:38.101 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:37 smithi169 ceph-mon[33465]: from='client.57245 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104"}]': finished 2024-09-16T08:57:38.101 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:37 smithi169 ceph-mon[33465]: osdmap e553: 8 total, 8 up, 8 in 2024-09-16T08:57:38.101 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:38.101 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:37 smithi169 ceph-mon[33465]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:38.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:37 smithi160 ceph-mon[30137]: pgmap v838: 164 pgs: 164 active+clean; 583 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-16T08:57:38.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:37 smithi160 ceph-mon[30137]: from='client.57245 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104"}]': finished 2024-09-16T08:57:38.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:37 smithi160 ceph-mon[30137]: osdmap e553: 8 total, 8 up, 8 in 2024-09-16T08:57:38.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:38.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:37 smithi160 ceph-mon[30137]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:38.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:37 smithi160 ceph-mon[40028]: pgmap v838: 164 pgs: 164 active+clean; 583 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-16T08:57:38.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:37 smithi160 ceph-mon[40028]: from='client.57245 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi160-76079-104"}]': finished 2024-09-16T08:57:38.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:37 smithi160 ceph-mon[40028]: osdmap e553: 8 total, 8 up, 8 in 2024-09-16T08:57:38.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1899831554' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:38.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:37 smithi160 ceph-mon[40028]: from='client.57245 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi160-76079-104"} : dispatch 2024-09-16T08:57:40.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:39 smithi169 ceph-mon[33465]: from='client.57245 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi160-76079-104"}]': finished 2024-09-16T08:57:40.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:39 smithi169 ceph-mon[33465]: osdmap e554: 8 total, 8 up, 8 in 2024-09-16T08:57:40.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:40.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:39 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:40.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:40.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:39 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:40.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-76079-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:57:40.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:39 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-76079-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:57:40.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:39 smithi169 ceph-mon[33465]: pgmap v841: 164 pgs: 164 active+clean; 583 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-16T08:57:40.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[30137]: from='client.57245 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi160-76079-104"}]': finished 2024-09-16T08:57:40.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[30137]: osdmap e554: 8 total, 8 up, 8 in 2024-09-16T08:57:40.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:40.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:40.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:40.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:40.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-76079-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:57:40.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-76079-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:57:40.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[30137]: pgmap v841: 164 pgs: 164 active+clean; 583 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-16T08:57:40.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[40028]: from='client.57245 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi160-76079-104"}]': finished 2024-09-16T08:57:40.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[40028]: osdmap e554: 8 total, 8 up, 8 in 2024-09-16T08:57:40.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:40.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:40.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:40.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:40.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-76079-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:57:40.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-76079-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-16T08:57:40.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:39 smithi160 ceph-mon[40028]: pgmap v841: 164 pgs: 164 active+clean; 583 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-16T08:57:41.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:40 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-76079-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:57:41.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:40 smithi169 ceph-mon[33465]: osdmap e555: 8 total, 8 up, 8 in 2024-09-16T08:57:41.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76079-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:41.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:40 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76079-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:41.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:40 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-76079-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:57:41.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:40 smithi160 ceph-mon[30137]: osdmap e555: 8 total, 8 up, 8 in 2024-09-16T08:57:41.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76079-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:41.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:40 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76079-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:41.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:40 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi160-76079-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-16T08:57:41.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:40 smithi160 ceph-mon[40028]: osdmap e555: 8 total, 8 up, 8 in 2024-09-16T08:57:41.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76079-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:41.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:40 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76079-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:42.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:41 smithi169 ceph-mon[33465]: osdmap e556: 8 total, 8 up, 8 in 2024-09-16T08:57:42.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:41 smithi169 ceph-mon[33465]: pgmap v844: 164 pgs: 164 active+clean; 583 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:42.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:41 smithi160 ceph-mon[40028]: osdmap e556: 8 total, 8 up, 8 in 2024-09-16T08:57:42.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:41 smithi160 ceph-mon[40028]: pgmap v844: 164 pgs: 164 active+clean; 583 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:42.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:41 smithi160 ceph-mon[30137]: osdmap e556: 8 total, 8 up, 8 in 2024-09-16T08:57:42.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:41 smithi160 ceph-mon[30137]: pgmap v844: 164 pgs: 164 active+clean; 583 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:57:43.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:42 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76079-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:57:43.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:42 smithi169 ceph-mon[33465]: osdmap e557: 8 total, 8 up, 8 in 2024-09-16T08:57:43.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:42 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76079-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:57:43.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:42 smithi160 ceph-mon[30137]: osdmap e557: 8 total, 8 up, 8 in 2024-09-16T08:57:43.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:42 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi160-76079-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:57:43.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:42 smithi160 ceph-mon[40028]: osdmap e557: 8 total, 8 up, 8 in 2024-09-16T08:57:44.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:43 smithi169 ceph-mon[33465]: osdmap e558: 8 total, 8 up, 8 in 2024-09-16T08:57:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:43 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:43 smithi169 ceph-mon[33465]: pgmap v847: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:43 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:57:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:43 smithi169 ceph-mon[33465]: osdmap e559: 8 total, 8 up, 8 in 2024-09-16T08:57:44.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:43 smithi160 ceph-mon[30137]: osdmap e558: 8 total, 8 up, 8 in 2024-09-16T08:57:44.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:44.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:43 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:44.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:43 smithi160 ceph-mon[30137]: pgmap v847: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:44.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:43 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:57:44.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:43 smithi160 ceph-mon[30137]: osdmap e559: 8 total, 8 up, 8 in 2024-09-16T08:57:44.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:43 smithi160 ceph-mon[40028]: osdmap e558: 8 total, 8 up, 8 in 2024-09-16T08:57:44.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:44.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:43 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:44.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:43 smithi160 ceph-mon[40028]: pgmap v847: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:44.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:43 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:57:44.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:43 smithi160 ceph-mon[40028]: osdmap e559: 8 total, 8 up, 8 in 2024-09-16T08:57:45.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:44 smithi169 ceph-mon[33465]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:45.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:57:45.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:44 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:57:45.101 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:44 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:57:45.101 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:44 smithi169 ceph-mon[33465]: osdmap e560: 8 total, 8 up, 8 in 2024-09-16T08:57:45.101 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-112"} : dispatch 2024-09-16T08:57:45.101 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:44 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-112"} : dispatch 2024-09-16T08:57:45.197 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:57:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:57:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:57:45.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:44 smithi160 ceph-mon[30137]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:45.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:57:45.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:44 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:57:45.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:44 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:57:45.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:44 smithi160 ceph-mon[30137]: osdmap e560: 8 total, 8 up, 8 in 2024-09-16T08:57:45.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-112"} : dispatch 2024-09-16T08:57:45.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:44 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-112"} : dispatch 2024-09-16T08:57:45.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:44 smithi160 ceph-mon[40028]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:45.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:57:45.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:44 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:57:45.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:44 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:57:45.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:44 smithi160 ceph-mon[40028]: osdmap e560: 8 total, 8 up, 8 in 2024-09-16T08:57:45.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-112"} : dispatch 2024-09-16T08:57:45.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:44 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-112"} : dispatch 2024-09-16T08:57:45.600 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:57:45 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:57:46.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:45 smithi169 ceph-mon[33465]: pgmap v850: 204 pgs: 204 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:57:46.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:45 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:46.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:45 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-112"}]': finished 2024-09-16T08:57:46.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:45 smithi169 ceph-mon[33465]: osdmap e561: 8 total, 8 up, 8 in 2024-09-16T08:57:46.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:45 smithi160 ceph-mon[40028]: pgmap v850: 204 pgs: 204 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:57:46.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:45 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:46.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:45 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-112"}]': finished 2024-09-16T08:57:46.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:45 smithi160 ceph-mon[40028]: osdmap e561: 8 total, 8 up, 8 in 2024-09-16T08:57:46.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:45 smithi160 ceph-mon[30137]: pgmap v850: 204 pgs: 204 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:57:46.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:45 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:46.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:45 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-112"}]': finished 2024-09-16T08:57:46.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:45 smithi160 ceph-mon[30137]: osdmap e561: 8 total, 8 up, 8 in 2024-09-16T08:57:47.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:47.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:46 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:47.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:47.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:46 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:47.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:47.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:46 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:48.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:47 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:57:48.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:47 smithi169 ceph-mon[33465]: osdmap e562: 8 total, 8 up, 8 in 2024-09-16T08:57:48.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112"} : dispatch 2024-09-16T08:57:48.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:47 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112"} : dispatch 2024-09-16T08:57:48.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:47 smithi169 ceph-mon[33465]: pgmap v853: 204 pgs: 204 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:57:48.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:47 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:57:48.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:47 smithi160 ceph-mon[30137]: osdmap e562: 8 total, 8 up, 8 in 2024-09-16T08:57:48.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112"} : dispatch 2024-09-16T08:57:48.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:47 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112"} : dispatch 2024-09-16T08:57:48.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:47 smithi160 ceph-mon[30137]: pgmap v853: 204 pgs: 204 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:57:48.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:47 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:57:48.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:47 smithi160 ceph-mon[40028]: osdmap e562: 8 total, 8 up, 8 in 2024-09-16T08:57:48.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112"} : dispatch 2024-09-16T08:57:48.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:47 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112"} : dispatch 2024-09-16T08:57:48.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:47 smithi160 ceph-mon[40028]: pgmap v853: 204 pgs: 204 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:57:49.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:48 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112"}]': finished 2024-09-16T08:57:49.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:48 smithi169 ceph-mon[33465]: osdmap e563: 8 total, 8 up, 8 in 2024-09-16T08:57:49.197 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:48 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112"}]': finished 2024-09-16T08:57:49.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:48 smithi160 ceph-mon[40028]: osdmap e563: 8 total, 8 up, 8 in 2024-09-16T08:57:49.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:48 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-112"}]': finished 2024-09-16T08:57:49.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:48 smithi160 ceph-mon[30137]: osdmap e563: 8 total, 8 up, 8 in 2024-09-16T08:57:50.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:49 smithi160 ceph-mon[30137]: osdmap e564: 8 total, 8 up, 8 in 2024-09-16T08:57:50.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:57:50.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:49 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:50.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:49 smithi160 ceph-mon[30137]: pgmap v856: 172 pgs: 172 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T08:57:50.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:49 smithi160 ceph-mon[40028]: osdmap e564: 8 total, 8 up, 8 in 2024-09-16T08:57:50.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:57:50.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:49 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:50.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:49 smithi160 ceph-mon[40028]: pgmap v856: 172 pgs: 172 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T08:57:50.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:49 smithi169 ceph-mon[33465]: osdmap e564: 8 total, 8 up, 8 in 2024-09-16T08:57:50.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:57:50.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:49 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:57:50.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:49 smithi169 ceph-mon[33465]: pgmap v856: 172 pgs: 172 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T08:57:51.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:50 smithi160 ceph-mon[30137]: osdmap e565: 8 total, 8 up, 8 in 2024-09-16T08:57:51.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:51.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:50 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:51.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:50 smithi160 ceph-mon[40028]: osdmap e565: 8 total, 8 up, 8 in 2024-09-16T08:57:51.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:51.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:50 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:51.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:50 smithi169 ceph-mon[33465]: osdmap e565: 8 total, 8 up, 8 in 2024-09-16T08:57:51.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:51.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:50 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:52.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:51 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:57:52.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:51 smithi160 ceph-mon[30137]: osdmap e566: 8 total, 8 up, 8 in 2024-09-16T08:57:52.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:57:52.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:51 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:57:52.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:51 smithi160 ceph-mon[30137]: pgmap v859: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T08:57:52.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:51 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:57:52.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:51 smithi160 ceph-mon[40028]: osdmap e566: 8 total, 8 up, 8 in 2024-09-16T08:57:52.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:57:52.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:51 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:57:52.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:51 smithi160 ceph-mon[40028]: pgmap v859: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T08:57:52.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:51 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:57:52.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:51 smithi169 ceph-mon[33465]: osdmap e566: 8 total, 8 up, 8 in 2024-09-16T08:57:52.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:57:52.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:51 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:57:52.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:51 smithi169 ceph-mon[33465]: pgmap v859: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T08:57:53.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:52 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:57:53.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:52 smithi160 ceph-mon[30137]: osdmap e567: 8 total, 8 up, 8 in 2024-09-16T08:57:53.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-114"} : dispatch 2024-09-16T08:57:53.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:52 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-114"} : dispatch 2024-09-16T08:57:53.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:52 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:57:53.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:52 smithi160 ceph-mon[40028]: osdmap e567: 8 total, 8 up, 8 in 2024-09-16T08:57:53.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-114"} : dispatch 2024-09-16T08:57:53.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:52 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-114"} : dispatch 2024-09-16T08:57:53.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:52 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:57:53.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:52 smithi169 ceph-mon[33465]: osdmap e567: 8 total, 8 up, 8 in 2024-09-16T08:57:53.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-114"} : dispatch 2024-09-16T08:57:53.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:52 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-114"} : dispatch 2024-09-16T08:57:54.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:53 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-114"}]': finished 2024-09-16T08:57:54.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:53 smithi160 ceph-mon[30137]: osdmap e568: 8 total, 8 up, 8 in 2024-09-16T08:57:54.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-114", "mode": "writeback"} : dispatch 2024-09-16T08:57:54.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:53 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-114", "mode": "writeback"} : dispatch 2024-09-16T08:57:54.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:53 smithi160 ceph-mon[30137]: pgmap v862: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:54.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:53 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-114"}]': finished 2024-09-16T08:57:54.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:53 smithi160 ceph-mon[40028]: osdmap e568: 8 total, 8 up, 8 in 2024-09-16T08:57:54.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-114", "mode": "writeback"} : dispatch 2024-09-16T08:57:54.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:53 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-114", "mode": "writeback"} : dispatch 2024-09-16T08:57:54.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:53 smithi160 ceph-mon[40028]: pgmap v862: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:54.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:53 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-114"}]': finished 2024-09-16T08:57:54.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:53 smithi169 ceph-mon[33465]: osdmap e568: 8 total, 8 up, 8 in 2024-09-16T08:57:54.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-114", "mode": "writeback"} : dispatch 2024-09-16T08:57:54.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:53 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-114", "mode": "writeback"} : dispatch 2024-09-16T08:57:54.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:53 smithi169 ceph-mon[33465]: pgmap v862: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:57:55.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:57:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:57:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:57:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:54 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:57:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:54 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-114", "mode": "writeback"}]': finished 2024-09-16T08:57:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:54 smithi160 ceph-mon[30137]: osdmap e569: 8 total, 8 up, 8 in 2024-09-16T08:57:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:55.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:54 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:55.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:54 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:57:55.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:54 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-114", "mode": "writeback"}]': finished 2024-09-16T08:57:55.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:54 smithi160 ceph-mon[40028]: osdmap e569: 8 total, 8 up, 8 in 2024-09-16T08:57:55.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:55.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:54 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:55.332 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:54 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:57:55.333 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:54 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-114", "mode": "writeback"}]': finished 2024-09-16T08:57:55.333 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:54 smithi169 ceph-mon[33465]: osdmap e569: 8 total, 8 up, 8 in 2024-09-16T08:57:55.333 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:55.333 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:54 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:57:55.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:57:55 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:57:56.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:57:56.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[30137]: osdmap e570: 8 total, 8 up, 8 in 2024-09-16T08:57:56.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114"} : dispatch 2024-09-16T08:57:56.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114"} : dispatch 2024-09-16T08:57:56.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[30137]: pgmap v865: 204 pgs: 204 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-16T08:57:56.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:56.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:57:56.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114"}]': finished 2024-09-16T08:57:56.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[30137]: osdmap e571: 8 total, 8 up, 8 in 2024-09-16T08:57:56.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:57:56.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[40028]: osdmap e570: 8 total, 8 up, 8 in 2024-09-16T08:57:56.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114"} : dispatch 2024-09-16T08:57:56.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114"} : dispatch 2024-09-16T08:57:56.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[40028]: pgmap v865: 204 pgs: 204 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-16T08:57:56.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:56.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:57:56.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114"}]': finished 2024-09-16T08:57:56.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:55 smithi160 ceph-mon[40028]: osdmap e571: 8 total, 8 up, 8 in 2024-09-16T08:57:56.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:55 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:57:56.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:55 smithi169 ceph-mon[33465]: osdmap e570: 8 total, 8 up, 8 in 2024-09-16T08:57:56.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114"} : dispatch 2024-09-16T08:57:56.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:55 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114"} : dispatch 2024-09-16T08:57:56.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:55 smithi169 ceph-mon[33465]: pgmap v865: 204 pgs: 204 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-16T08:57:56.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:55 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:57:56.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:55 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:57:56.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:55 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-114"}]': finished 2024-09-16T08:57:56.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:55 smithi169 ceph-mon[33465]: osdmap e571: 8 total, 8 up, 8 in 2024-09-16T08:57:58.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:57 smithi160 ceph-mon[30137]: osdmap e572: 8 total, 8 up, 8 in 2024-09-16T08:57:58.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:57 smithi160 ceph-mon[30137]: pgmap v868: 172 pgs: 172 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:57:58.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:57 smithi160 ceph-mon[40028]: osdmap e572: 8 total, 8 up, 8 in 2024-09-16T08:57:58.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:57 smithi160 ceph-mon[40028]: pgmap v868: 172 pgs: 172 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:57:58.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:57 smithi169 ceph-mon[33465]: osdmap e572: 8 total, 8 up, 8 in 2024-09-16T08:57:58.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:57 smithi169 ceph-mon[33465]: pgmap v868: 172 pgs: 172 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:57:59.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:58 smithi160 ceph-mon[30137]: osdmap e573: 8 total, 8 up, 8 in 2024-09-16T08:57:59.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:59.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:58 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:59.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:58 smithi160 ceph-mon[40028]: osdmap e573: 8 total, 8 up, 8 in 2024-09-16T08:57:59.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:59.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:58 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:59.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:58 smithi169 ceph-mon[33465]: osdmap e573: 8 total, 8 up, 8 in 2024-09-16T08:57:59.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:57:59.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:58 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:00.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:59 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:58:00.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:59 smithi160 ceph-mon[30137]: osdmap e574: 8 total, 8 up, 8 in 2024-09-16T08:58:00.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:57:59 smithi160 ceph-mon[30137]: pgmap v871: 204 pgs: 204 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-09-16T08:58:00.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:59 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:58:00.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:59 smithi160 ceph-mon[40028]: osdmap e574: 8 total, 8 up, 8 in 2024-09-16T08:58:00.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:57:59 smithi160 ceph-mon[40028]: pgmap v871: 204 pgs: 204 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-09-16T08:58:00.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:59 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:58:00.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:59 smithi169 ceph-mon[33465]: osdmap e574: 8 total, 8 up, 8 in 2024-09-16T08:58:00.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:57:59 smithi169 ceph-mon[33465]: pgmap v871: 204 pgs: 204 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-09-16T08:58:01.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:00 smithi160 ceph-mon[30137]: osdmap e575: 8 total, 8 up, 8 in 2024-09-16T08:58:01.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:01.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:00 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:01.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:00 smithi160 ceph-mon[40028]: osdmap e575: 8 total, 8 up, 8 in 2024-09-16T08:58:01.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:01.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:00 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:01.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:00 smithi169 ceph-mon[33465]: osdmap e575: 8 total, 8 up, 8 in 2024-09-16T08:58:01.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:01.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:00 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:02.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:01 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:58:02.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:01 smithi160 ceph-mon[30137]: osdmap e576: 8 total, 8 up, 8 in 2024-09-16T08:58:02.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-116"} : dispatch 2024-09-16T08:58:02.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:01 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-116"} : dispatch 2024-09-16T08:58:02.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:01 smithi160 ceph-mon[30137]: pgmap v874: 204 pgs: 204 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:58:02.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:01 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-116"}]': finished 2024-09-16T08:58:02.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:01 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:58:02.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:01 smithi160 ceph-mon[40028]: osdmap e576: 8 total, 8 up, 8 in 2024-09-16T08:58:02.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-116"} : dispatch 2024-09-16T08:58:02.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:01 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-116"} : dispatch 2024-09-16T08:58:02.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:01 smithi160 ceph-mon[40028]: pgmap v874: 204 pgs: 204 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:58:02.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:01 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-116"}]': finished 2024-09-16T08:58:02.251 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:01 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:58:02.251 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:01 smithi169 ceph-mon[33465]: osdmap e576: 8 total, 8 up, 8 in 2024-09-16T08:58:02.251 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-116"} : dispatch 2024-09-16T08:58:02.251 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:01 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-116"} : dispatch 2024-09-16T08:58:02.251 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:01 smithi169 ceph-mon[33465]: pgmap v874: 204 pgs: 204 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T08:58:02.251 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:01 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-116"}]': finished 2024-09-16T08:58:03.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:02 smithi169 ceph-mon[33465]: osdmap e577: 8 total, 8 up, 8 in 2024-09-16T08:58:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:02 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-116", "mode": "writeback"} : dispatch 2024-09-16T08:58:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:02 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-116", "mode": "writeback"} : dispatch 2024-09-16T08:58:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:02 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:58:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:02 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-116", "mode": "writeback"}]': finished 2024-09-16T08:58:03.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:02 smithi169 ceph-mon[33465]: osdmap e578: 8 total, 8 up, 8 in 2024-09-16T08:58:03.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:02 smithi160 ceph-mon[30137]: osdmap e577: 8 total, 8 up, 8 in 2024-09-16T08:58:03.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:02 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-116", "mode": "writeback"} : dispatch 2024-09-16T08:58:03.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:02 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-116", "mode": "writeback"} : dispatch 2024-09-16T08:58:03.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:02 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:58:03.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:02 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-116", "mode": "writeback"}]': finished 2024-09-16T08:58:03.449 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:02 smithi160 ceph-mon[30137]: osdmap e578: 8 total, 8 up, 8 in 2024-09-16T08:58:03.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:02 smithi160 ceph-mon[40028]: osdmap e577: 8 total, 8 up, 8 in 2024-09-16T08:58:03.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:02 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-116", "mode": "writeback"} : dispatch 2024-09-16T08:58:03.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:02 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-116", "mode": "writeback"} : dispatch 2024-09-16T08:58:03.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:02 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:58:03.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:02 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-116", "mode": "writeback"}]': finished 2024-09-16T08:58:03.450 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:02 smithi160 ceph-mon[40028]: osdmap e578: 8 total, 8 up, 8 in 2024-09-16T08:58:04.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "325.2"} : dispatch 2024-09-16T08:58:04.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:03 smithi169 ceph-mon[33465]: from='mon.? v1:172.21.15.160:0/2040510614' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.2"}]: dispatch 2024-09-16T08:58:04.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:03 smithi169 ceph-mon[33465]: pgmap v877: 204 pgs: 204 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:58:04.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "325.2"} : dispatch 2024-09-16T08:58:04.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:03 smithi160 ceph-mon[30137]: from='mon.? v1:172.21.15.160:0/2040510614' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.2"}]: dispatch 2024-09-16T08:58:04.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:03 smithi160 ceph-mon[30137]: pgmap v877: 204 pgs: 204 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:58:04.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "325.2"} : dispatch 2024-09-16T08:58:04.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:03 smithi160 ceph-mon[40028]: from='mon.? v1:172.21.15.160:0/2040510614' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.2"}]: dispatch 2024-09-16T08:58:04.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:03 smithi160 ceph-mon[40028]: pgmap v877: 204 pgs: 204 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:58:05.344 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:05 smithi169 ceph-mon[33465]: 325.2 scrub starts 2024-09-16T08:58:05.344 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:05 smithi169 ceph-mon[33465]: 325.2 scrub ok 2024-09-16T08:58:05.344 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:05 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:05.344 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:05 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:58:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:05 smithi160 ceph-mon[30137]: 325.2 scrub starts 2024-09-16T08:58:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:05 smithi160 ceph-mon[30137]: 325.2 scrub ok 2024-09-16T08:58:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:05 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:05.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:05 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:58:05.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:58:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:58:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:58:05.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:05 smithi160 ceph-mon[40028]: 325.2 scrub starts 2024-09-16T08:58:05.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:05 smithi160 ceph-mon[40028]: 325.2 scrub ok 2024-09-16T08:58:05.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:05 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:05.449 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:05 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:58:05.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:58:05 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:58:06.349 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:06 smithi169 ceph-mon[33465]: pgmap v878: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 576 B/s wr, 2 op/s 2024-09-16T08:58:06.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:06.447 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:06 smithi160 ceph-mon[30137]: pgmap v878: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 576 B/s wr, 2 op/s 2024-09-16T08:58:06.448 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:06.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:06 smithi160 ceph-mon[40028]: pgmap v878: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 576 B/s wr, 2 op/s 2024-09-16T08:58:06.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:08 smithi169 ceph-mon[33465]: pgmap v879: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 485 B/s wr, 2 op/s 2024-09-16T08:58:08.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:08 smithi160 ceph-mon[30137]: pgmap v879: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 485 B/s wr, 2 op/s 2024-09-16T08:58:08.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:08 smithi160 ceph-mon[40028]: pgmap v879: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 485 B/s wr, 2 op/s 2024-09-16T08:58:10.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:10 smithi169 ceph-mon[33465]: pgmap v880: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-09-16T08:58:10.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:10 smithi160 ceph-mon[30137]: pgmap v880: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-09-16T08:58:10.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:10 smithi160 ceph-mon[40028]: pgmap v880: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-09-16T08:58:12.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:12 smithi169 ceph-mon[33465]: pgmap v881: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 329 B/s wr, 1 op/s 2024-09-16T08:58:12.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:12 smithi160 ceph-mon[30137]: pgmap v881: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 329 B/s wr, 1 op/s 2024-09-16T08:58:12.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:12 smithi160 ceph-mon[40028]: pgmap v881: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 329 B/s wr, 1 op/s 2024-09-16T08:58:14.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:14 smithi169 ceph-mon[33465]: pgmap v882: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 297 B/s wr, 1 op/s 2024-09-16T08:58:14.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:14 smithi160 ceph-mon[30137]: pgmap v882: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 297 B/s wr, 1 op/s 2024-09-16T08:58:14.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:14 smithi160 ceph-mon[40028]: pgmap v882: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 297 B/s wr, 1 op/s 2024-09-16T08:58:15.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:58:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:58:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:58:15.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:58:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:58:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:16 smithi169 ceph-mon[33465]: pgmap v883: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:16.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:16.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:16 smithi160 ceph-mon[30137]: pgmap v883: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:16 smithi160 ceph-mon[40028]: pgmap v883: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:18 smithi169 ceph-mon[33465]: pgmap v884: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:58:18.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:18 smithi160 ceph-mon[30137]: pgmap v884: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:58:18.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:18 smithi160 ceph-mon[40028]: pgmap v884: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:58:19.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:19 smithi169 ceph-mon[33465]: osdmap e579: 8 total, 8 up, 8 in 2024-09-16T08:58:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:19 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:58:19.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:19 smithi160 ceph-mon[30137]: osdmap e579: 8 total, 8 up, 8 in 2024-09-16T08:58:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:19 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:58:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:19 smithi160 ceph-mon[40028]: osdmap e579: 8 total, 8 up, 8 in 2024-09-16T08:58:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:19 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:58:20.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:20 smithi160 ceph-mon[30137]: pgmap v886: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:58:20.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:20 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:58:20.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:20 smithi160 ceph-mon[30137]: osdmap e580: 8 total, 8 up, 8 in 2024-09-16T08:58:20.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116"} : dispatch 2024-09-16T08:58:20.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:20 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116"} : dispatch 2024-09-16T08:58:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:20 smithi160 ceph-mon[40028]: pgmap v886: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:58:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:20 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:58:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:20 smithi160 ceph-mon[40028]: osdmap e580: 8 total, 8 up, 8 in 2024-09-16T08:58:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116"} : dispatch 2024-09-16T08:58:20.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:20 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116"} : dispatch 2024-09-16T08:58:20.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:20 smithi169 ceph-mon[33465]: pgmap v886: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T08:58:20.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:20 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:58:20.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:20 smithi169 ceph-mon[33465]: osdmap e580: 8 total, 8 up, 8 in 2024-09-16T08:58:20.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116"} : dispatch 2024-09-16T08:58:20.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:20 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116"} : dispatch 2024-09-16T08:58:21.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:21 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:58:21.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:21 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116"}]': finished 2024-09-16T08:58:21.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:21 smithi160 ceph-mon[30137]: osdmap e581: 8 total, 8 up, 8 in 2024-09-16T08:58:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:21 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:58:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:21 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116"}]': finished 2024-09-16T08:58:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:21 smithi160 ceph-mon[40028]: osdmap e581: 8 total, 8 up, 8 in 2024-09-16T08:58:21.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:21 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:58:21.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:21 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-116"}]': finished 2024-09-16T08:58:21.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:21 smithi169 ceph-mon[33465]: osdmap e581: 8 total, 8 up, 8 in 2024-09-16T08:58:22.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:22 smithi160 ceph-mon[30137]: pgmap v889: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-16T08:58:22.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:22 smithi160 ceph-mon[30137]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:22.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:22 smithi160 ceph-mon[30137]: osdmap e582: 8 total, 8 up, 8 in 2024-09-16T08:58:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:22 smithi160 ceph-mon[40028]: pgmap v889: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-16T08:58:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:22 smithi160 ceph-mon[40028]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:22 smithi160 ceph-mon[40028]: osdmap e582: 8 total, 8 up, 8 in 2024-09-16T08:58:22.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:22 smithi169 ceph-mon[33465]: pgmap v889: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-16T08:58:22.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:22 smithi169 ceph-mon[33465]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:22.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:22 smithi169 ceph-mon[33465]: osdmap e582: 8 total, 8 up, 8 in 2024-09-16T08:58:23.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:23 smithi160 ceph-mon[30137]: osdmap e583: 8 total, 8 up, 8 in 2024-09-16T08:58:23.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:23 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:23.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:23 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:23.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:23 smithi160 ceph-mon[40028]: osdmap e583: 8 total, 8 up, 8 in 2024-09-16T08:58:23.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:23 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:23.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:23 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:23.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:23 smithi169 ceph-mon[33465]: osdmap e583: 8 total, 8 up, 8 in 2024-09-16T08:58:23.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:23 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:23.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:23 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:24.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:24 smithi160 ceph-mon[40028]: pgmap v892: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:58:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:24 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:58:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:24 smithi160 ceph-mon[40028]: osdmap e584: 8 total, 8 up, 8 in 2024-09-16T08:58:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:24 smithi160 ceph-mon[40028]: osdmap e585: 8 total, 8 up, 8 in 2024-09-16T08:58:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:24 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:24 smithi160 ceph-mon[30137]: pgmap v892: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:58:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:24 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:58:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:24 smithi160 ceph-mon[30137]: osdmap e584: 8 total, 8 up, 8 in 2024-09-16T08:58:24.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:24 smithi160 ceph-mon[30137]: osdmap e585: 8 total, 8 up, 8 in 2024-09-16T08:58:24.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:24.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:24 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:24.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:24 smithi169 ceph-mon[33465]: pgmap v892: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:58:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:24 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:58:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:24 smithi169 ceph-mon[33465]: osdmap e584: 8 total, 8 up, 8 in 2024-09-16T08:58:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:24 smithi169 ceph-mon[33465]: osdmap e585: 8 total, 8 up, 8 in 2024-09-16T08:58:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:24 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:25.376 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:58:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:58:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:58:25.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:25 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:58:25.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:25 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:58:25.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:25 smithi160 ceph-mon[30137]: osdmap e586: 8 total, 8 up, 8 in 2024-09-16T08:58:25.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-118"} : dispatch 2024-09-16T08:58:25.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:25 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-118"} : dispatch 2024-09-16T08:58:25.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:25 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:58:25.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:25 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:58:25.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:25 smithi160 ceph-mon[40028]: osdmap e586: 8 total, 8 up, 8 in 2024-09-16T08:58:25.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-118"} : dispatch 2024-09-16T08:58:25.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:25 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-118"} : dispatch 2024-09-16T08:58:25.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:58:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:58:25.776 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:25 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:58:25.776 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:25 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:58:25.776 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:25 smithi169 ceph-mon[33465]: osdmap e586: 8 total, 8 up, 8 in 2024-09-16T08:58:25.776 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-118"} : dispatch 2024-09-16T08:58:25.777 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:25 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-118"} : dispatch 2024-09-16T08:58:26.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:26 smithi160 ceph-mon[40028]: pgmap v896: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 261 B/s wr, 1 op/s 2024-09-16T08:58:26.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:26 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-118"}]': finished 2024-09-16T08:58:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:26 smithi160 ceph-mon[40028]: osdmap e587: 8 total, 8 up, 8 in 2024-09-16T08:58:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:26 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-118", "mode": "writeback"} : dispatch 2024-09-16T08:58:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:26 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-118", "mode": "writeback"} : dispatch 2024-09-16T08:58:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:26 smithi160 ceph-mon[30137]: pgmap v896: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 261 B/s wr, 1 op/s 2024-09-16T08:58:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:26 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-118"}]': finished 2024-09-16T08:58:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:26 smithi160 ceph-mon[30137]: osdmap e587: 8 total, 8 up, 8 in 2024-09-16T08:58:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:26 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-118", "mode": "writeback"} : dispatch 2024-09-16T08:58:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:26 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-118", "mode": "writeback"} : dispatch 2024-09-16T08:58:26.713 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:26 smithi169 ceph-mon[33465]: pgmap v896: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 261 B/s wr, 1 op/s 2024-09-16T08:58:26.713 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:26.713 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:26 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-118"}]': finished 2024-09-16T08:58:26.713 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:26 smithi169 ceph-mon[33465]: osdmap e587: 8 total, 8 up, 8 in 2024-09-16T08:58:26.713 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:26 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-118", "mode": "writeback"} : dispatch 2024-09-16T08:58:26.713 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:26 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-118", "mode": "writeback"} : dispatch 2024-09-16T08:58:27.642 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:27 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:58:27.642 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:27 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-118", "mode": "writeback"}]': finished 2024-09-16T08:58:27.642 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:27 smithi160 ceph-mon[30137]: osdmap e588: 8 total, 8 up, 8 in 2024-09-16T08:58:27.642 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:27 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:58:27.642 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:27 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-118", "mode": "writeback"}]': finished 2024-09-16T08:58:27.642 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:27 smithi160 ceph-mon[40028]: osdmap e588: 8 total, 8 up, 8 in 2024-09-16T08:58:27.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:27 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:58:27.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:27 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-118", "mode": "writeback"}]': finished 2024-09-16T08:58:27.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:27 smithi169 ceph-mon[33465]: osdmap e588: 8 total, 8 up, 8 in 2024-09-16T08:58:28.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:28 smithi169 ceph-mon[33465]: pgmap v899: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 262 B/s wr, 1 op/s 2024-09-16T08:58:28.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:28 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:28 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:28 smithi169 ceph-mon[33465]: osdmap e589: 8 total, 8 up, 8 in 2024-09-16T08:58:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:28.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:28 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:28 smithi160 ceph-mon[30137]: pgmap v899: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 262 B/s wr, 1 op/s 2024-09-16T08:58:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:28 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:28 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:28 smithi160 ceph-mon[30137]: osdmap e589: 8 total, 8 up, 8 in 2024-09-16T08:58:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:28 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:28 smithi160 ceph-mon[40028]: pgmap v899: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 262 B/s wr, 1 op/s 2024-09-16T08:58:28.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:28 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:28.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:28 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:28.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:28 smithi160 ceph-mon[40028]: osdmap e589: 8 total, 8 up, 8 in 2024-09-16T08:58:28.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:28.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:28 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:29.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:29 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:29 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:58:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:29 smithi160 ceph-mon[30137]: osdmap e590: 8 total, 8 up, 8 in 2024-09-16T08:58:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118"} : dispatch 2024-09-16T08:58:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:29 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118"} : dispatch 2024-09-16T08:58:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:29 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:29.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:29 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:29.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:29 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:29.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:29 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:58:29.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:29 smithi160 ceph-mon[40028]: osdmap e590: 8 total, 8 up, 8 in 2024-09-16T08:58:29.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118"} : dispatch 2024-09-16T08:58:29.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:29 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118"} : dispatch 2024-09-16T08:58:29.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:29 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:29.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:29 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:29.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:29 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:29.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:29 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:58:29.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:29 smithi169 ceph-mon[33465]: osdmap e590: 8 total, 8 up, 8 in 2024-09-16T08:58:29.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118"} : dispatch 2024-09-16T08:58:29.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:29 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118"} : dispatch 2024-09-16T08:58:29.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:29 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:29.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:29 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:30 smithi160 ceph-mon[40028]: pgmap v901: 204 pgs: 204 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-16T08:58:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:30 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:58:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:30 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:58:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:30 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:30 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:58:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:30 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118"}]': finished 2024-09-16T08:58:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:30 smithi160 ceph-mon[40028]: osdmap e591: 8 total, 8 up, 8 in 2024-09-16T08:58:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:30 smithi160 ceph-mon[30137]: pgmap v901: 204 pgs: 204 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-16T08:58:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:30 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:58:30.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:30 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:58:30.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:30 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:30.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:30 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:58:30.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:30 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118"}]': finished 2024-09-16T08:58:30.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:30 smithi160 ceph-mon[30137]: osdmap e591: 8 total, 8 up, 8 in 2024-09-16T08:58:30.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:30 smithi169 ceph-mon[33465]: pgmap v901: 204 pgs: 204 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-16T08:58:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:30 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:58:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:30 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:58:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:30 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:30 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:58:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:30 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-118"}]': finished 2024-09-16T08:58:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:30 smithi169 ceph-mon[33465]: osdmap e591: 8 total, 8 up, 8 in 2024-09-16T08:58:32.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:32 smithi169 ceph-mon[33465]: pgmap v904: 204 pgs: 204 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:32.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:32 smithi169 ceph-mon[33465]: osdmap e592: 8 total, 8 up, 8 in 2024-09-16T08:58:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:32 smithi160 ceph-mon[30137]: pgmap v904: 204 pgs: 204 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:32 smithi160 ceph-mon[30137]: osdmap e592: 8 total, 8 up, 8 in 2024-09-16T08:58:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:32 smithi160 ceph-mon[40028]: pgmap v904: 204 pgs: 204 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:32 smithi160 ceph-mon[40028]: osdmap e592: 8 total, 8 up, 8 in 2024-09-16T08:58:33.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:33 smithi169 ceph-mon[33465]: osdmap e593: 8 total, 8 up, 8 in 2024-09-16T08:58:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:33.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:33 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:33 smithi160 ceph-mon[30137]: osdmap e593: 8 total, 8 up, 8 in 2024-09-16T08:58:33.716 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:33.716 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:33 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:33.716 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:33 smithi160 ceph-mon[40028]: osdmap e593: 8 total, 8 up, 8 in 2024-09-16T08:58:33.716 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:33.716 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:33 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:34 smithi169 ceph-mon[33465]: pgmap v907: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:58:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:34 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:58:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:34 smithi169 ceph-mon[33465]: osdmap e594: 8 total, 8 up, 8 in 2024-09-16T08:58:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:34 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:58:34.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:34 smithi169 ceph-mon[33465]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:34 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:58:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:34 smithi169 ceph-mon[33465]: osdmap e595: 8 total, 8 up, 8 in 2024-09-16T08:58:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-120"} : dispatch 2024-09-16T08:58:34.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:34 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-120"} : dispatch 2024-09-16T08:58:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[40028]: pgmap v907: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:58:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:58:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[40028]: osdmap e594: 8 total, 8 up, 8 in 2024-09-16T08:58:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:58:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[40028]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:58:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[40028]: osdmap e595: 8 total, 8 up, 8 in 2024-09-16T08:58:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-120"} : dispatch 2024-09-16T08:58:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-120"} : dispatch 2024-09-16T08:58:34.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[30137]: pgmap v907: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:58:34.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:58:34.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[30137]: osdmap e594: 8 total, 8 up, 8 in 2024-09-16T08:58:34.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:34.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:34.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:58:34.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:58:34.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[30137]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:34.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:58:34.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[30137]: osdmap e595: 8 total, 8 up, 8 in 2024-09-16T08:58:34.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-120"} : dispatch 2024-09-16T08:58:34.700 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-120"} : dispatch 2024-09-16T08:58:35.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:58:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:58:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:58:35.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:58:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:58:36.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:36 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-120"}]': finished 2024-09-16T08:58:36.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:36 smithi169 ceph-mon[33465]: osdmap e596: 8 total, 8 up, 8 in 2024-09-16T08:58:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-120", "mode": "writeback"} : dispatch 2024-09-16T08:58:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:36 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-120", "mode": "writeback"} : dispatch 2024-09-16T08:58:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:36 smithi169 ceph-mon[33465]: pgmap v911: 204 pgs: 204 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 256 B/s wr, 1 op/s 2024-09-16T08:58:36.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:36 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:36.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:36 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-120"}]': finished 2024-09-16T08:58:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:36 smithi160 ceph-mon[40028]: osdmap e596: 8 total, 8 up, 8 in 2024-09-16T08:58:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-120", "mode": "writeback"} : dispatch 2024-09-16T08:58:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:36 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-120", "mode": "writeback"} : dispatch 2024-09-16T08:58:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:36 smithi160 ceph-mon[40028]: pgmap v911: 204 pgs: 204 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 256 B/s wr, 1 op/s 2024-09-16T08:58:36.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:36 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:36 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-120"}]': finished 2024-09-16T08:58:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:36 smithi160 ceph-mon[30137]: osdmap e596: 8 total, 8 up, 8 in 2024-09-16T08:58:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-120", "mode": "writeback"} : dispatch 2024-09-16T08:58:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:36 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-120", "mode": "writeback"} : dispatch 2024-09-16T08:58:36.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:36 smithi160 ceph-mon[30137]: pgmap v911: 204 pgs: 204 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 256 B/s wr, 1 op/s 2024-09-16T08:58:36.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:36 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:37.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:37 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:58:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:37 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-120", "mode": "writeback"}]': finished 2024-09-16T08:58:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:37 smithi169 ceph-mon[33465]: osdmap e597: 8 total, 8 up, 8 in 2024-09-16T08:58:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:37.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:37 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:37.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:37 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:58:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:37 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-120", "mode": "writeback"}]': finished 2024-09-16T08:58:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:37 smithi160 ceph-mon[40028]: osdmap e597: 8 total, 8 up, 8 in 2024-09-16T08:58:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:37 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:37 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:58:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:37 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-120", "mode": "writeback"}]': finished 2024-09-16T08:58:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:37 smithi160 ceph-mon[30137]: osdmap e597: 8 total, 8 up, 8 in 2024-09-16T08:58:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:37 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:38.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:38 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:58:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:38 smithi169 ceph-mon[33465]: pgmap v914: 204 pgs: 204 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 257 B/s wr, 1 op/s 2024-09-16T08:58:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:38 smithi169 ceph-mon[33465]: osdmap e598: 8 total, 8 up, 8 in 2024-09-16T08:58:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120"} : dispatch 2024-09-16T08:58:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:38 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120"} : dispatch 2024-09-16T08:58:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:38 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:58:38.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:38 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120"}]': finished 2024-09-16T08:58:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:38 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:58:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:38 smithi160 ceph-mon[30137]: pgmap v914: 204 pgs: 204 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 257 B/s wr, 1 op/s 2024-09-16T08:58:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:38 smithi160 ceph-mon[30137]: osdmap e598: 8 total, 8 up, 8 in 2024-09-16T08:58:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120"} : dispatch 2024-09-16T08:58:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:38 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120"} : dispatch 2024-09-16T08:58:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:38 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:58:38.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:38 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120"}]': finished 2024-09-16T08:58:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:38 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:58:38.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:38 smithi160 ceph-mon[40028]: pgmap v914: 204 pgs: 204 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 257 B/s wr, 1 op/s 2024-09-16T08:58:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:38 smithi160 ceph-mon[40028]: osdmap e598: 8 total, 8 up, 8 in 2024-09-16T08:58:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120"} : dispatch 2024-09-16T08:58:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:38 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120"} : dispatch 2024-09-16T08:58:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:38 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:58:38.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:38 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-120"}]': finished 2024-09-16T08:58:39.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:39 smithi169 ceph-mon[33465]: osdmap e599: 8 total, 8 up, 8 in 2024-09-16T08:58:39.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:39 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:39.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:39 smithi160 ceph-mon[30137]: osdmap e599: 8 total, 8 up, 8 in 2024-09-16T08:58:39.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:39 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:39.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:39 smithi160 ceph-mon[40028]: osdmap e599: 8 total, 8 up, 8 in 2024-09-16T08:58:39.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:39 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:40.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:40 smithi169 ceph-mon[33465]: pgmap v916: 204 pgs: 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 510 B/s wr, 3 op/s 2024-09-16T08:58:40.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:40 smithi169 ceph-mon[33465]: osdmap e600: 8 total, 8 up, 8 in 2024-09-16T08:58:40.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:40 smithi160 ceph-mon[30137]: pgmap v916: 204 pgs: 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 510 B/s wr, 3 op/s 2024-09-16T08:58:40.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:40 smithi160 ceph-mon[30137]: osdmap e600: 8 total, 8 up, 8 in 2024-09-16T08:58:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:40 smithi160 ceph-mon[40028]: pgmap v916: 204 pgs: 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 510 B/s wr, 3 op/s 2024-09-16T08:58:40.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:40 smithi160 ceph-mon[40028]: osdmap e600: 8 total, 8 up, 8 in 2024-09-16T08:58:41.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:41 smithi169 ceph-mon[33465]: osdmap e601: 8 total, 8 up, 8 in 2024-09-16T08:58:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:41.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:41 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:41.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:41 smithi160 ceph-mon[30137]: osdmap e601: 8 total, 8 up, 8 in 2024-09-16T08:58:41.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:41.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:41 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:41.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:41 smithi160 ceph-mon[40028]: osdmap e601: 8 total, 8 up, 8 in 2024-09-16T08:58:41.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:41.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:41 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:42.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:42 smithi169 ceph-mon[33465]: pgmap v919: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:42 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:58:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:42 smithi169 ceph-mon[33465]: osdmap e602: 8 total, 8 up, 8 in 2024-09-16T08:58:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:42.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:42 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:42.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:42 smithi160 ceph-mon[30137]: pgmap v919: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:42 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:58:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:42 smithi160 ceph-mon[30137]: osdmap e602: 8 total, 8 up, 8 in 2024-09-16T08:58:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:42.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:42 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:42 smithi160 ceph-mon[40028]: pgmap v919: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:42 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:58:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:42 smithi160 ceph-mon[40028]: osdmap e602: 8 total, 8 up, 8 in 2024-09-16T08:58:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:42.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:42 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:43 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:58:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:43 smithi169 ceph-mon[33465]: osdmap e603: 8 total, 8 up, 8 in 2024-09-16T08:58:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-122"} : dispatch 2024-09-16T08:58:43.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:43 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-122"} : dispatch 2024-09-16T08:58:43.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:43 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:58:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:43 smithi160 ceph-mon[30137]: osdmap e603: 8 total, 8 up, 8 in 2024-09-16T08:58:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-122"} : dispatch 2024-09-16T08:58:43.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:43 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-122"} : dispatch 2024-09-16T08:58:43.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:43 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:58:43.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:43 smithi160 ceph-mon[40028]: osdmap e603: 8 total, 8 up, 8 in 2024-09-16T08:58:43.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-122"} : dispatch 2024-09-16T08:58:43.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:43 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-122"} : dispatch 2024-09-16T08:58:44.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:44 smithi169 ceph-mon[33465]: pgmap v922: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:58:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:44 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-122"}]': finished 2024-09-16T08:58:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:44 smithi169 ceph-mon[33465]: osdmap e604: 8 total, 8 up, 8 in 2024-09-16T08:58:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-122", "mode": "writeback"} : dispatch 2024-09-16T08:58:44.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:44 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-122", "mode": "writeback"} : dispatch 2024-09-16T08:58:44.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:44 smithi160 ceph-mon[40028]: pgmap v922: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:58:44.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:44 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-122"}]': finished 2024-09-16T08:58:44.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:44 smithi160 ceph-mon[40028]: osdmap e604: 8 total, 8 up, 8 in 2024-09-16T08:58:44.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-122", "mode": "writeback"} : dispatch 2024-09-16T08:58:44.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:44 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-122", "mode": "writeback"} : dispatch 2024-09-16T08:58:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:44 smithi160 ceph-mon[30137]: pgmap v922: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:58:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:44 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-122"}]': finished 2024-09-16T08:58:44.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:44 smithi160 ceph-mon[30137]: osdmap e604: 8 total, 8 up, 8 in 2024-09-16T08:58:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-122", "mode": "writeback"} : dispatch 2024-09-16T08:58:44.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:44 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-122", "mode": "writeback"} : dispatch 2024-09-16T08:58:45.339 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:58:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:58:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:58:45.600 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:58:45 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:58:45.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:45 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:58:45.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:45 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-122", "mode": "writeback"}]': finished 2024-09-16T08:58:45.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:45 smithi169 ceph-mon[33465]: osdmap e605: 8 total, 8 up, 8 in 2024-09-16T08:58:45.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:45 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:45.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:45 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:45.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:45 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:58:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:45 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-122", "mode": "writeback"}]': finished 2024-09-16T08:58:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:45 smithi160 ceph-mon[40028]: osdmap e605: 8 total, 8 up, 8 in 2024-09-16T08:58:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:45 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:45.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:45 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:45 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:58:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:45 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-122", "mode": "writeback"}]': finished 2024-09-16T08:58:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:45 smithi160 ceph-mon[30137]: osdmap e605: 8 total, 8 up, 8 in 2024-09-16T08:58:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:45 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:45.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:45 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:46 smithi169 ceph-mon[33465]: pgmap v925: 204 pgs: 204 active+clean; 583 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:46 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:58:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:46 smithi169 ceph-mon[33465]: osdmap e606: 8 total, 8 up, 8 in 2024-09-16T08:58:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122"} : dispatch 2024-09-16T08:58:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:46 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122"} : dispatch 2024-09-16T08:58:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:46 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:46 smithi160 ceph-mon[40028]: pgmap v925: 204 pgs: 204 active+clean; 583 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:46 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:58:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:46 smithi160 ceph-mon[40028]: osdmap e606: 8 total, 8 up, 8 in 2024-09-16T08:58:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122"} : dispatch 2024-09-16T08:58:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:46 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122"} : dispatch 2024-09-16T08:58:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:46 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:46 smithi160 ceph-mon[30137]: pgmap v925: 204 pgs: 204 active+clean; 583 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:46 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:58:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:46 smithi160 ceph-mon[30137]: osdmap e606: 8 total, 8 up, 8 in 2024-09-16T08:58:46.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122"} : dispatch 2024-09-16T08:58:46.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:46 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122"} : dispatch 2024-09-16T08:58:46.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:46 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:47.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:47 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:58:47.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:47 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122"}]': finished 2024-09-16T08:58:47.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:47 smithi160 ceph-mon[30137]: osdmap e607: 8 total, 8 up, 8 in 2024-09-16T08:58:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:47 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:58:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:47 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122"}]': finished 2024-09-16T08:58:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:47 smithi160 ceph-mon[40028]: osdmap e607: 8 total, 8 up, 8 in 2024-09-16T08:58:47.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:47 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:58:47.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:47 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-122"}]': finished 2024-09-16T08:58:47.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:47 smithi169 ceph-mon[33465]: osdmap e607: 8 total, 8 up, 8 in 2024-09-16T08:58:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:48 smithi160 ceph-mon[30137]: pgmap v928: 204 pgs: 204 active+clean; 583 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:48 smithi160 ceph-mon[30137]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:48 smithi160 ceph-mon[30137]: osdmap e608: 8 total, 8 up, 8 in 2024-09-16T08:58:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:48 smithi160 ceph-mon[40028]: pgmap v928: 204 pgs: 204 active+clean; 583 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:48 smithi160 ceph-mon[40028]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:48 smithi160 ceph-mon[40028]: osdmap e608: 8 total, 8 up, 8 in 2024-09-16T08:58:48.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:48 smithi169 ceph-mon[33465]: pgmap v928: 204 pgs: 204 active+clean; 583 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:48 smithi169 ceph-mon[33465]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:48 smithi169 ceph-mon[33465]: osdmap e608: 8 total, 8 up, 8 in 2024-09-16T08:58:49.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:49 smithi160 ceph-mon[30137]: osdmap e609: 8 total, 8 up, 8 in 2024-09-16T08:58:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:49 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:58:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:49 smithi160 ceph-mon[40028]: osdmap e609: 8 total, 8 up, 8 in 2024-09-16T08:58:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:49 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:58:49.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:49 smithi169 ceph-mon[33465]: osdmap e609: 8 total, 8 up, 8 in 2024-09-16T08:58:49.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:49.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:49 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:58:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:50 smithi160 ceph-mon[30137]: pgmap v931: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:50 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:58:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:50 smithi160 ceph-mon[30137]: osdmap e610: 8 total, 8 up, 8 in 2024-09-16T08:58:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:50 smithi160 ceph-mon[40028]: pgmap v931: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:50 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:58:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:50 smithi160 ceph-mon[40028]: osdmap e610: 8 total, 8 up, 8 in 2024-09-16T08:58:50.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:50 smithi169 ceph-mon[33465]: pgmap v931: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:50.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:50 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:58:50.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:50 smithi169 ceph-mon[33465]: osdmap e610: 8 total, 8 up, 8 in 2024-09-16T08:58:51.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:51 smithi160 ceph-mon[30137]: osdmap e611: 8 total, 8 up, 8 in 2024-09-16T08:58:51.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:51.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:51 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:51.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:51 smithi160 ceph-mon[40028]: osdmap e611: 8 total, 8 up, 8 in 2024-09-16T08:58:51.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:51.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:51 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:51.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:51 smithi169 ceph-mon[33465]: osdmap e611: 8 total, 8 up, 8 in 2024-09-16T08:58:51.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:51.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:51 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:58:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:52 smithi160 ceph-mon[30137]: pgmap v934: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:52 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:58:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:52 smithi160 ceph-mon[30137]: osdmap e612: 8 total, 8 up, 8 in 2024-09-16T08:58:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-124"} : dispatch 2024-09-16T08:58:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:52 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-124"} : dispatch 2024-09-16T08:58:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:52 smithi160 ceph-mon[40028]: pgmap v934: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:52 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:58:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:52 smithi160 ceph-mon[40028]: osdmap e612: 8 total, 8 up, 8 in 2024-09-16T08:58:52.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-124"} : dispatch 2024-09-16T08:58:52.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:52 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-124"} : dispatch 2024-09-16T08:58:52.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:52 smithi169 ceph-mon[33465]: pgmap v934: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:58:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:52 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:58:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:52 smithi169 ceph-mon[33465]: osdmap e612: 8 total, 8 up, 8 in 2024-09-16T08:58:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-124"} : dispatch 2024-09-16T08:58:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:52 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-124"} : dispatch 2024-09-16T08:58:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:53 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-124"}]': finished 2024-09-16T08:58:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:53 smithi160 ceph-mon[30137]: osdmap e613: 8 total, 8 up, 8 in 2024-09-16T08:58:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-124", "mode": "writeback"} : dispatch 2024-09-16T08:58:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:53 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-124", "mode": "writeback"} : dispatch 2024-09-16T08:58:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:53 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-124"}]': finished 2024-09-16T08:58:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:53 smithi160 ceph-mon[40028]: osdmap e613: 8 total, 8 up, 8 in 2024-09-16T08:58:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-124", "mode": "writeback"} : dispatch 2024-09-16T08:58:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:53 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-124", "mode": "writeback"} : dispatch 2024-09-16T08:58:53.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:53 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-124"}]': finished 2024-09-16T08:58:53.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:53 smithi169 ceph-mon[33465]: osdmap e613: 8 total, 8 up, 8 in 2024-09-16T08:58:53.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-124", "mode": "writeback"} : dispatch 2024-09-16T08:58:53.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:53 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-124", "mode": "writeback"} : dispatch 2024-09-16T08:58:54.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:54 smithi160 ceph-mon[30137]: pgmap v937: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:58:54.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:54 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:58:54.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:54 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-124", "mode": "writeback"}]': finished 2024-09-16T08:58:54.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:54 smithi160 ceph-mon[30137]: osdmap e614: 8 total, 8 up, 8 in 2024-09-16T08:58:54.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:54 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:54 smithi160 ceph-mon[40028]: pgmap v937: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:58:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:54 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:58:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:54 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-124", "mode": "writeback"}]': finished 2024-09-16T08:58:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:54 smithi160 ceph-mon[40028]: osdmap e614: 8 total, 8 up, 8 in 2024-09-16T08:58:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:54 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:54.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:54 smithi169 ceph-mon[33465]: pgmap v937: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:58:54.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:54 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:58:54.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:54 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-124", "mode": "writeback"}]': finished 2024-09-16T08:58:54.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:54 smithi169 ceph-mon[33465]: osdmap e614: 8 total, 8 up, 8 in 2024-09-16T08:58:54.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:54 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:55.417 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:58:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:58:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:58:55.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:55 smithi160 ceph-mon[30137]: osdmap e615: 8 total, 8 up, 8 in 2024-09-16T08:58:55.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:55.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:55 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:55.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:55 smithi160 ceph-mon[40028]: osdmap e615: 8 total, 8 up, 8 in 2024-09-16T08:58:55.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:55.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:55 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:55.850 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:58:55 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:58:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:55 smithi169 ceph-mon[33465]: osdmap e615: 8 total, 8 up, 8 in 2024-09-16T08:58:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:55 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:58:56.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:56 smithi160 ceph-mon[30137]: pgmap v940: 204 pgs: 204 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T08:58:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:56 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:56 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:58:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:56 smithi160 ceph-mon[30137]: osdmap e616: 8 total, 8 up, 8 in 2024-09-16T08:58:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124"} : dispatch 2024-09-16T08:58:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:56 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124"} : dispatch 2024-09-16T08:58:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:56 smithi160 ceph-mon[40028]: pgmap v940: 204 pgs: 204 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T08:58:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:56 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:56 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:58:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:56 smithi160 ceph-mon[40028]: osdmap e616: 8 total, 8 up, 8 in 2024-09-16T08:58:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124"} : dispatch 2024-09-16T08:58:56.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:56 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124"} : dispatch 2024-09-16T08:58:56.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:56 smithi169 ceph-mon[33465]: pgmap v940: 204 pgs: 204 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T08:58:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:56 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:58:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:56 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:58:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:56 smithi169 ceph-mon[33465]: osdmap e616: 8 total, 8 up, 8 in 2024-09-16T08:58:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124"} : dispatch 2024-09-16T08:58:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:56 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124"} : dispatch 2024-09-16T08:58:57.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:57 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:58:57.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:57 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124"}]': finished 2024-09-16T08:58:57.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:57 smithi160 ceph-mon[30137]: osdmap e617: 8 total, 8 up, 8 in 2024-09-16T08:58:57.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:57 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:58:57.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:57 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124"}]': finished 2024-09-16T08:58:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:57 smithi160 ceph-mon[40028]: osdmap e617: 8 total, 8 up, 8 in 2024-09-16T08:58:57.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:57 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:58:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:57 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-124"}]': finished 2024-09-16T08:58:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:57 smithi169 ceph-mon[33465]: osdmap e617: 8 total, 8 up, 8 in 2024-09-16T08:58:58.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:58 smithi160 ceph-mon[30137]: pgmap v943: 204 pgs: 204 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T08:58:58.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:58 smithi160 ceph-mon[30137]: osdmap e618: 8 total, 8 up, 8 in 2024-09-16T08:58:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:58 smithi160 ceph-mon[40028]: pgmap v943: 204 pgs: 204 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T08:58:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:58 smithi160 ceph-mon[40028]: osdmap e618: 8 total, 8 up, 8 in 2024-09-16T08:58:58.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:58 smithi169 ceph-mon[33465]: pgmap v943: 204 pgs: 204 active+clean; 583 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-16T08:58:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:58 smithi169 ceph-mon[33465]: osdmap e618: 8 total, 8 up, 8 in 2024-09-16T08:58:59.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:59 smithi169 ceph-mon[33465]: osdmap e619: 8 total, 8 up, 8 in 2024-09-16T08:58:59.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:59.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:59 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:59.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:58:59 smithi169 ceph-mon[33465]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:59.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:59 smithi160 ceph-mon[30137]: osdmap e619: 8 total, 8 up, 8 in 2024-09-16T08:58:59.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:59.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:59 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:59.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:58:59 smithi160 ceph-mon[30137]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:58:59.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:59 smithi160 ceph-mon[40028]: osdmap e619: 8 total, 8 up, 8 in 2024-09-16T08:58:59.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:59.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:59 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:58:59.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:58:59 smithi160 ceph-mon[40028]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:59:00.849 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:00 smithi160 ceph-mon[30137]: pgmap v946: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:00.849 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:00 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:00.850 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:00 smithi160 ceph-mon[30137]: osdmap e620: 8 total, 8 up, 8 in 2024-09-16T08:59:00.850 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:00.850 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:00 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:00.850 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:00 smithi160 ceph-mon[40028]: pgmap v946: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:00.850 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:00 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:00.850 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:00 smithi160 ceph-mon[40028]: osdmap e620: 8 total, 8 up, 8 in 2024-09-16T08:59:00.850 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:00.850 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:00 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:00.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:00 smithi169 ceph-mon[33465]: pgmap v946: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:00.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:00 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:00.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:00 smithi169 ceph-mon[33465]: osdmap e620: 8 total, 8 up, 8 in 2024-09-16T08:59:00.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:00.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:00 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:01.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:01 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:01.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:01 smithi169 ceph-mon[33465]: osdmap e621: 8 total, 8 up, 8 in 2024-09-16T08:59:01.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-126"} : dispatch 2024-09-16T08:59:01.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:01 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-126"} : dispatch 2024-09-16T08:59:01.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:01 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-126"}]': finished 2024-09-16T08:59:01.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:01 smithi169 ceph-mon[33465]: osdmap e622: 8 total, 8 up, 8 in 2024-09-16T08:59:01.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-126", "mode": "writeback"} : dispatch 2024-09-16T08:59:01.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:01 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-126", "mode": "writeback"} : dispatch 2024-09-16T08:59:01.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:01 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:01.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:01 smithi160 ceph-mon[30137]: osdmap e621: 8 total, 8 up, 8 in 2024-09-16T08:59:01.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-126"} : dispatch 2024-09-16T08:59:01.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:01 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-126"} : dispatch 2024-09-16T08:59:01.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:01 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-126"}]': finished 2024-09-16T08:59:01.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:01 smithi160 ceph-mon[30137]: osdmap e622: 8 total, 8 up, 8 in 2024-09-16T08:59:01.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-126", "mode": "writeback"} : dispatch 2024-09-16T08:59:01.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:01 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-126", "mode": "writeback"} : dispatch 2024-09-16T08:59:01.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:01 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:01.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:01 smithi160 ceph-mon[40028]: osdmap e621: 8 total, 8 up, 8 in 2024-09-16T08:59:01.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-126"} : dispatch 2024-09-16T08:59:01.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:01 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-126"} : dispatch 2024-09-16T08:59:01.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:01 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-126"}]': finished 2024-09-16T08:59:01.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:01 smithi160 ceph-mon[40028]: osdmap e622: 8 total, 8 up, 8 in 2024-09-16T08:59:01.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-126", "mode": "writeback"} : dispatch 2024-09-16T08:59:01.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:01 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-126", "mode": "writeback"} : dispatch 2024-09-16T08:59:02.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:02 smithi169 ceph-mon[33465]: pgmap v949: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:02.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:02 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:02.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:02 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-126", "mode": "writeback"}]': finished 2024-09-16T08:59:02.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:02 smithi169 ceph-mon[33465]: osdmap e623: 8 total, 8 up, 8 in 2024-09-16T08:59:02.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:02 smithi160 ceph-mon[30137]: pgmap v949: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:02.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:02 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:02.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:02 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-126", "mode": "writeback"}]': finished 2024-09-16T08:59:02.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:02 smithi160 ceph-mon[30137]: osdmap e623: 8 total, 8 up, 8 in 2024-09-16T08:59:02.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:02 smithi160 ceph-mon[40028]: pgmap v949: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:02.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:02 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:02.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:02 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-126", "mode": "writeback"}]': finished 2024-09-16T08:59:02.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:02 smithi160 ceph-mon[40028]: osdmap e623: 8 total, 8 up, 8 in 2024-09-16T08:59:03.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:03 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:03.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:03.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:03 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:03.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:03.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:03 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:04 smithi169 ceph-mon[33465]: pgmap v952: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:04 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:04 smithi169 ceph-mon[33465]: osdmap e624: 8 total, 8 up, 8 in 2024-09-16T08:59:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126"} : dispatch 2024-09-16T08:59:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:04 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126"} : dispatch 2024-09-16T08:59:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:59:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:04 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:59:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:04 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:04.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:04 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126"}]': finished 2024-09-16T08:59:04.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:04 smithi169 ceph-mon[33465]: osdmap e625: 8 total, 8 up, 8 in 2024-09-16T08:59:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[30137]: pgmap v952: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[30137]: osdmap e624: 8 total, 8 up, 8 in 2024-09-16T08:59:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126"} : dispatch 2024-09-16T08:59:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126"} : dispatch 2024-09-16T08:59:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:59:04.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:04.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:59:04.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:04.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126"}]': finished 2024-09-16T08:59:04.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[30137]: osdmap e625: 8 total, 8 up, 8 in 2024-09-16T08:59:04.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[40028]: pgmap v952: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:04.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:04.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[40028]: osdmap e624: 8 total, 8 up, 8 in 2024-09-16T08:59:04.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126"} : dispatch 2024-09-16T08:59:04.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126"} : dispatch 2024-09-16T08:59:04.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:59:04.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:04.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:59:04.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:04.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-126"}]': finished 2024-09-16T08:59:04.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-mon[40028]: osdmap e625: 8 total, 8 up, 8 in 2024-09-16T08:59:05.273 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:59:05.273 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9036 ms) 2024-09-16T08:59:05.273 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-09-16T08:59:05.274 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:59:05.274 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6053 ms) 2024-09-16T08:59:05.274 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-09-16T08:59:05.274 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:59:05.274 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5044 ms) 2024-09-16T08:59:05.274 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-09-16T08:59:05.274 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:59:05.274 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25504 ms) 2024-09-16T08:59:05.274 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-09-16T08:59:05.274 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:59:05.274 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14054 ms) 2024-09-16T08:59:05.274 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-09-16T08:59:05.274 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12077 ms) 2024-09-16T08:59:05.274 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-09-16T08:59:05.274 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-09-16T08:59:05.274 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: 00000010 69 73 |is| 2024-09-16T08:59:05.275 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: 00000012 2024-09-16T08:59:05.275 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12064 ms) 2024-09-16T08:59:05.275 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-09-16T08:59:05.275 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-16T08:59:05.275 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6041 ms) 2024-09-16T08:59:05.275 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (557432 ms total) 2024-09-16T08:59:05.275 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: 2024-09-16T08:59:05.275 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-09-16T08:59:05.275 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-09-16T08:59:05.275 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1018 ms) 2024-09-16T08:59:05.275 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-09-16T08:59:05.275 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11070 ms) 2024-09-16T08:59:05.275 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-09-16T08:59:05.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18092 ms) 2024-09-16T08:59:05.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-09-16T08:59:05.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-09-16T08:59:05.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30183 ms total) 2024-09-16T08:59:05.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: 2024-09-16T08:59:05.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-09-16T08:59:05.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-09-16T08:59:05.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7050 ms) 2024-09-16T08:59:05.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-09-16T08:59:05.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8072 ms) 2024-09-16T08:59:05.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-09-16T08:59:05.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: waiting for scrub... 2024-09-16T08:59:05.276 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: done waiting 2024-09-16T08:59:05.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24453 ms) 2024-09-16T08:59:05.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-09-16T08:59:05.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9920 ms) 2024-09-16T08:59:05.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-09-16T08:59:05.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (8008 ms) 2024-09-16T08:59:05.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-09-16T08:59:05.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8067 ms) 2024-09-16T08:59:05.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-09-16T08:59:05.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10075 ms) 2024-09-16T08:59:05.277 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-09-16T08:59:05.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:59:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:59:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:59:05.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:59:05 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:59:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:06 smithi169 ceph-mon[33465]: osdmap e626: 8 total, 8 up, 8 in 2024-09-16T08:59:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:06 smithi169 ceph-mon[33465]: pgmap v956: 172 pgs: 172 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 267 B/s wr, 1 op/s 2024-09-16T08:59:06.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:06.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:06 smithi160 ceph-mon[30137]: osdmap e626: 8 total, 8 up, 8 in 2024-09-16T08:59:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:06 smithi160 ceph-mon[30137]: pgmap v956: 172 pgs: 172 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 267 B/s wr, 1 op/s 2024-09-16T08:59:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:06 smithi160 ceph-mon[40028]: osdmap e626: 8 total, 8 up, 8 in 2024-09-16T08:59:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:06 smithi160 ceph-mon[40028]: pgmap v956: 172 pgs: 172 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 267 B/s wr, 1 op/s 2024-09-16T08:59:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:07.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:07 smithi169 ceph-mon[33465]: osdmap e627: 8 total, 8 up, 8 in 2024-09-16T08:59:07.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:07 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:07.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:07 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:07.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:07 smithi160 ceph-mon[30137]: osdmap e627: 8 total, 8 up, 8 in 2024-09-16T08:59:07.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:07 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:07.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:07 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:07.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:07 smithi160 ceph-mon[40028]: osdmap e627: 8 total, 8 up, 8 in 2024-09-16T08:59:07.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:07 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:07.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:07 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:08 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:08 smithi169 ceph-mon[33465]: osdmap e628: 8 total, 8 up, 8 in 2024-09-16T08:59:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:08 smithi169 ceph-mon[33465]: pgmap v959: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 269 B/s wr, 1 op/s 2024-09-16T08:59:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:08 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:08.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:08 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:08 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:08 smithi160 ceph-mon[30137]: osdmap e628: 8 total, 8 up, 8 in 2024-09-16T08:59:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:08 smithi160 ceph-mon[30137]: pgmap v959: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 269 B/s wr, 1 op/s 2024-09-16T08:59:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:08 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:08.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:08 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:08.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:08 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:08.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:08 smithi160 ceph-mon[40028]: osdmap e628: 8 total, 8 up, 8 in 2024-09-16T08:59:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:08 smithi160 ceph-mon[40028]: pgmap v959: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 269 B/s wr, 1 op/s 2024-09-16T08:59:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:08 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:08.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:08 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:09.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:09 smithi169 ceph-mon[33465]: osdmap e629: 8 total, 8 up, 8 in 2024-09-16T08:59:09.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-128"} : dispatch 2024-09-16T08:59:09.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:09 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-128"} : dispatch 2024-09-16T08:59:09.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:09 smithi160 ceph-mon[30137]: osdmap e629: 8 total, 8 up, 8 in 2024-09-16T08:59:09.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-128"} : dispatch 2024-09-16T08:59:09.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:09 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-128"} : dispatch 2024-09-16T08:59:09.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:09 smithi160 ceph-mon[40028]: osdmap e629: 8 total, 8 up, 8 in 2024-09-16T08:59:09.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-128"} : dispatch 2024-09-16T08:59:09.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:09 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-128"} : dispatch 2024-09-16T08:59:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:10 smithi169 ceph-mon[33465]: pgmap v961: 204 pgs: 204 active+clean; 583 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:59:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:10 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-128"}]': finished 2024-09-16T08:59:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:10 smithi169 ceph-mon[33465]: osdmap e630: 8 total, 8 up, 8 in 2024-09-16T08:59:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-128", "mode": "writeback"} : dispatch 2024-09-16T08:59:10.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:10 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-128", "mode": "writeback"} : dispatch 2024-09-16T08:59:10.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:10 smithi160 ceph-mon[30137]: pgmap v961: 204 pgs: 204 active+clean; 583 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:59:10.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:10 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-128"}]': finished 2024-09-16T08:59:10.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:10 smithi160 ceph-mon[30137]: osdmap e630: 8 total, 8 up, 8 in 2024-09-16T08:59:10.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-128", "mode": "writeback"} : dispatch 2024-09-16T08:59:10.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:10 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-128", "mode": "writeback"} : dispatch 2024-09-16T08:59:10.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:10 smithi160 ceph-mon[40028]: pgmap v961: 204 pgs: 204 active+clean; 583 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:59:10.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:10 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-128"}]': finished 2024-09-16T08:59:10.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:10 smithi160 ceph-mon[40028]: osdmap e630: 8 total, 8 up, 8 in 2024-09-16T08:59:10.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-128", "mode": "writeback"} : dispatch 2024-09-16T08:59:10.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:10 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-128", "mode": "writeback"} : dispatch 2024-09-16T08:59:11.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:11 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:11 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-128", "mode": "writeback"}]': finished 2024-09-16T08:59:11.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:11 smithi169 ceph-mon[33465]: osdmap e631: 8 total, 8 up, 8 in 2024-09-16T08:59:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:11 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:11 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-128", "mode": "writeback"}]': finished 2024-09-16T08:59:11.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:11 smithi160 ceph-mon[30137]: osdmap e631: 8 total, 8 up, 8 in 2024-09-16T08:59:11.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:11 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:11.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:11 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-128", "mode": "writeback"}]': finished 2024-09-16T08:59:11.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:11 smithi160 ceph-mon[40028]: osdmap e631: 8 total, 8 up, 8 in 2024-09-16T08:59:12.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:12 smithi169 ceph-mon[33465]: pgmap v964: 204 pgs: 204 active+clean; 583 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:59:12.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:12 smithi160 ceph-mon[30137]: pgmap v964: 204 pgs: 204 active+clean; 583 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:59:12.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:12 smithi160 ceph-mon[40028]: pgmap v964: 204 pgs: 204 active+clean; 583 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:59:14.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:14 smithi169 ceph-mon[33465]: pgmap v965: 204 pgs: 204 active+clean; 583 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:59:14.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:14 smithi160 ceph-mon[30137]: pgmap v965: 204 pgs: 204 active+clean; 583 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:59:14.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:14 smithi160 ceph-mon[40028]: pgmap v965: 204 pgs: 204 active+clean; 583 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:59:15.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:59:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:59:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:59:15.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:59:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:59:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:16 smithi169 ceph-mon[33465]: pgmap v966: 204 pgs: 204 active+clean; 583 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 292 B/s wr, 1 op/s 2024-09-16T08:59:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:16 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:16.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:16.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:16 smithi160 ceph-mon[30137]: pgmap v966: 204 pgs: 204 active+clean; 583 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 292 B/s wr, 1 op/s 2024-09-16T08:59:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:16 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:16 smithi160 ceph-mon[40028]: pgmap v966: 204 pgs: 204 active+clean; 583 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 292 B/s wr, 1 op/s 2024-09-16T08:59:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:16 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:17.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:17 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:17 smithi169 ceph-mon[33465]: osdmap e632: 8 total, 8 up, 8 in 2024-09-16T08:59:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128"} : dispatch 2024-09-16T08:59:17.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:17 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128"} : dispatch 2024-09-16T08:59:17.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:17 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:17.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:17 smithi160 ceph-mon[30137]: osdmap e632: 8 total, 8 up, 8 in 2024-09-16T08:59:17.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128"} : dispatch 2024-09-16T08:59:17.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:17 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128"} : dispatch 2024-09-16T08:59:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:17 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:17 smithi160 ceph-mon[40028]: osdmap e632: 8 total, 8 up, 8 in 2024-09-16T08:59:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128"} : dispatch 2024-09-16T08:59:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:17 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128"} : dispatch 2024-09-16T08:59:18.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:18 smithi169 ceph-mon[33465]: pgmap v968: 204 pgs: 204 active+clean; 583 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 768 B/s rd, 256 B/s wr, 1 op/s 2024-09-16T08:59:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:18 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:18 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128"}]': finished 2024-09-16T08:59:18.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:18 smithi169 ceph-mon[33465]: osdmap e633: 8 total, 8 up, 8 in 2024-09-16T08:59:18.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:18 smithi160 ceph-mon[30137]: pgmap v968: 204 pgs: 204 active+clean; 583 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 768 B/s rd, 256 B/s wr, 1 op/s 2024-09-16T08:59:18.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:18 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:18.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:18 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128"}]': finished 2024-09-16T08:59:18.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:18 smithi160 ceph-mon[30137]: osdmap e633: 8 total, 8 up, 8 in 2024-09-16T08:59:18.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:18 smithi160 ceph-mon[40028]: pgmap v968: 204 pgs: 204 active+clean; 583 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 768 B/s rd, 256 B/s wr, 1 op/s 2024-09-16T08:59:18.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:18 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:18.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:18 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-128"}]': finished 2024-09-16T08:59:18.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:18 smithi160 ceph-mon[40028]: osdmap e633: 8 total, 8 up, 8 in 2024-09-16T08:59:19.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:19 smithi169 ceph-mon[33465]: osdmap e634: 8 total, 8 up, 8 in 2024-09-16T08:59:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:59:19.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:19 smithi160 ceph-mon[30137]: osdmap e634: 8 total, 8 up, 8 in 2024-09-16T08:59:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:59:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:19 smithi160 ceph-mon[40028]: osdmap e634: 8 total, 8 up, 8 in 2024-09-16T08:59:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:59:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:20.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:20 smithi169 ceph-mon[33465]: pgmap v971: 172 pgs: 172 active+clean; 583 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T08:59:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:20 smithi169 ceph-mon[33465]: osdmap e635: 8 total, 8 up, 8 in 2024-09-16T08:59:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:20.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:20 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:20.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:20 smithi160 ceph-mon[30137]: pgmap v971: 172 pgs: 172 active+clean; 583 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T08:59:20.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:20 smithi160 ceph-mon[30137]: osdmap e635: 8 total, 8 up, 8 in 2024-09-16T08:59:20.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:20.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:20 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:20 smithi160 ceph-mon[40028]: pgmap v971: 172 pgs: 172 active+clean; 583 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T08:59:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:20 smithi160 ceph-mon[40028]: osdmap e635: 8 total, 8 up, 8 in 2024-09-16T08:59:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:20 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:21.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:21 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:21.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:21 smithi160 ceph-mon[30137]: osdmap e636: 8 total, 8 up, 8 in 2024-09-16T08:59:21.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:21 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:21.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:21 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:21 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:21 smithi160 ceph-mon[40028]: osdmap e636: 8 total, 8 up, 8 in 2024-09-16T08:59:21.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:21 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:21.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:21 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:21.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:21 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:21.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:21 smithi169 ceph-mon[33465]: osdmap e636: 8 total, 8 up, 8 in 2024-09-16T08:59:21.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:21 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:21.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:21 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:22.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:22 smithi160 ceph-mon[30137]: pgmap v974: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:59:22.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:22 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:22.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:22 smithi160 ceph-mon[30137]: osdmap e637: 8 total, 8 up, 8 in 2024-09-16T08:59:22.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:22 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-130"} : dispatch 2024-09-16T08:59:22.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:22 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-130"} : dispatch 2024-09-16T08:59:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:22 smithi160 ceph-mon[40028]: pgmap v974: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:59:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:22 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:22 smithi160 ceph-mon[40028]: osdmap e637: 8 total, 8 up, 8 in 2024-09-16T08:59:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:22 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-130"} : dispatch 2024-09-16T08:59:22.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:22 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-130"} : dispatch 2024-09-16T08:59:22.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:22 smithi169 ceph-mon[33465]: pgmap v974: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:59:22.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:22 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:22.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:22 smithi169 ceph-mon[33465]: osdmap e637: 8 total, 8 up, 8 in 2024-09-16T08:59:22.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:22 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-130"} : dispatch 2024-09-16T08:59:22.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:22 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-130"} : dispatch 2024-09-16T08:59:23.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:23 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-130"}]': finished 2024-09-16T08:59:23.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:23 smithi160 ceph-mon[30137]: osdmap e638: 8 total, 8 up, 8 in 2024-09-16T08:59:23.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:23 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-130", "mode": "writeback"} : dispatch 2024-09-16T08:59:23.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:23 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-130", "mode": "writeback"} : dispatch 2024-09-16T08:59:23.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:23 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-130"}]': finished 2024-09-16T08:59:23.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:23 smithi160 ceph-mon[40028]: osdmap e638: 8 total, 8 up, 8 in 2024-09-16T08:59:23.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:23 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-130", "mode": "writeback"} : dispatch 2024-09-16T08:59:23.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:23 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-130", "mode": "writeback"} : dispatch 2024-09-16T08:59:23.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:23 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-130"}]': finished 2024-09-16T08:59:23.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:23 smithi169 ceph-mon[33465]: osdmap e638: 8 total, 8 up, 8 in 2024-09-16T08:59:23.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:23 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-130", "mode": "writeback"} : dispatch 2024-09-16T08:59:23.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:23 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-130", "mode": "writeback"} : dispatch 2024-09-16T08:59:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:24 smithi160 ceph-mon[30137]: pgmap v977: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 678 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:24 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:24 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-130", "mode": "writeback"}]': finished 2024-09-16T08:59:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:24 smithi160 ceph-mon[30137]: osdmap e639: 8 total, 8 up, 8 in 2024-09-16T08:59:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:24 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:24 smithi160 ceph-mon[40028]: pgmap v977: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 678 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:24 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:24 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-130", "mode": "writeback"}]': finished 2024-09-16T08:59:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:24 smithi160 ceph-mon[40028]: osdmap e639: 8 total, 8 up, 8 in 2024-09-16T08:59:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:24.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:24 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:24 smithi169 ceph-mon[33465]: pgmap v977: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 678 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:24 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:24 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-130", "mode": "writeback"}]': finished 2024-09-16T08:59:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:24 smithi169 ceph-mon[33465]: osdmap e639: 8 total, 8 up, 8 in 2024-09-16T08:59:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:24 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:25.382 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:59:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:59:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:59:25.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:25 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:25.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:25 smithi160 ceph-mon[30137]: osdmap e640: 8 total, 8 up, 8 in 2024-09-16T08:59:25.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130"} : dispatch 2024-09-16T08:59:25.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:25 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130"} : dispatch 2024-09-16T08:59:25.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:25 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:25.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:25 smithi160 ceph-mon[40028]: osdmap e640: 8 total, 8 up, 8 in 2024-09-16T08:59:25.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130"} : dispatch 2024-09-16T08:59:25.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:25 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130"} : dispatch 2024-09-16T08:59:25.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:59:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:59:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:25 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:25 smithi169 ceph-mon[33465]: osdmap e640: 8 total, 8 up, 8 in 2024-09-16T08:59:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130"} : dispatch 2024-09-16T08:59:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:25 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130"} : dispatch 2024-09-16T08:59:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:26 smithi160 ceph-mon[30137]: pgmap v980: 204 pgs: 204 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-16T08:59:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:26 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:26 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130"}]': finished 2024-09-16T08:59:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:26 smithi160 ceph-mon[30137]: osdmap e641: 8 total, 8 up, 8 in 2024-09-16T08:59:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:26 smithi160 ceph-mon[40028]: pgmap v980: 204 pgs: 204 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-16T08:59:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:26 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:26 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130"}]': finished 2024-09-16T08:59:26.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:26 smithi160 ceph-mon[40028]: osdmap e641: 8 total, 8 up, 8 in 2024-09-16T08:59:26.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:26.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:26 smithi169 ceph-mon[33465]: pgmap v980: 204 pgs: 204 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-16T08:59:26.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:26 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:26.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:26 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-130"}]': finished 2024-09-16T08:59:26.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:26 smithi169 ceph-mon[33465]: osdmap e641: 8 total, 8 up, 8 in 2024-09-16T08:59:26.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:27.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:27 smithi160 ceph-mon[30137]: osdmap e642: 8 total, 8 up, 8 in 2024-09-16T08:59:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:27 smithi160 ceph-mon[30137]: osdmap e643: 8 total, 8 up, 8 in 2024-09-16T08:59:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:27.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:27 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:27.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:27 smithi160 ceph-mon[40028]: osdmap e642: 8 total, 8 up, 8 in 2024-09-16T08:59:27.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:27 smithi160 ceph-mon[40028]: osdmap e643: 8 total, 8 up, 8 in 2024-09-16T08:59:27.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:27.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:27 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:27.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:27 smithi169 ceph-mon[33465]: osdmap e642: 8 total, 8 up, 8 in 2024-09-16T08:59:27.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:27 smithi169 ceph-mon[33465]: osdmap e643: 8 total, 8 up, 8 in 2024-09-16T08:59:27.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:27.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:27 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:28 smithi160 ceph-mon[30137]: pgmap v983: 172 pgs: 172 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:59:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:28 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:28 smithi160 ceph-mon[30137]: osdmap e644: 8 total, 8 up, 8 in 2024-09-16T08:59:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:28.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:28 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:28 smithi160 ceph-mon[40028]: pgmap v983: 172 pgs: 172 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:59:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:28 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:28.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:28 smithi160 ceph-mon[40028]: osdmap e644: 8 total, 8 up, 8 in 2024-09-16T08:59:28.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:28.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:28 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:28.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:28 smithi169 ceph-mon[33465]: pgmap v983: 172 pgs: 172 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T08:59:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:28 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:28 smithi169 ceph-mon[33465]: osdmap e644: 8 total, 8 up, 8 in 2024-09-16T08:59:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:28 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:30.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[30137]: pgmap v986: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:59:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[30137]: osdmap e645: 8 total, 8 up, 8 in 2024-09-16T08:59:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:59:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:59:30.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:59:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[40028]: pgmap v986: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:59:30.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:30.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[40028]: osdmap e645: 8 total, 8 up, 8 in 2024-09-16T08:59:30.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:30.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:30.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:59:30.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:30.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:59:30.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:30 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:59:30.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:30 smithi169 ceph-mon[33465]: pgmap v986: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:59:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:30 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:30 smithi169 ceph-mon[33465]: osdmap e645: 8 total, 8 up, 8 in 2024-09-16T08:59:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:30 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:30 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:59:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:30 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:30 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:59:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:30 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:59:31.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:31 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"}]': finished 2024-09-16T08:59:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:31 smithi160 ceph-mon[30137]: osdmap e646: 8 total, 8 up, 8 in 2024-09-16T08:59:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-132", "mode": "writeback"} : dispatch 2024-09-16T08:59:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:31 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-132", "mode": "writeback"} : dispatch 2024-09-16T08:59:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:31 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:31 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-132", "mode": "writeback"}]': finished 2024-09-16T08:59:31.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:31 smithi160 ceph-mon[30137]: osdmap e647: 8 total, 8 up, 8 in 2024-09-16T08:59:31.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:31 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"}]': finished 2024-09-16T08:59:31.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:31 smithi160 ceph-mon[40028]: osdmap e646: 8 total, 8 up, 8 in 2024-09-16T08:59:31.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-132", "mode": "writeback"} : dispatch 2024-09-16T08:59:31.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:31 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-132", "mode": "writeback"} : dispatch 2024-09-16T08:59:31.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:31 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:31.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:31 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-132", "mode": "writeback"}]': finished 2024-09-16T08:59:31.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:31 smithi160 ceph-mon[40028]: osdmap e647: 8 total, 8 up, 8 in 2024-09-16T08:59:31.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:31 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"}]': finished 2024-09-16T08:59:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:31 smithi169 ceph-mon[33465]: osdmap e646: 8 total, 8 up, 8 in 2024-09-16T08:59:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-132", "mode": "writeback"} : dispatch 2024-09-16T08:59:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:31 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-132", "mode": "writeback"} : dispatch 2024-09-16T08:59:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:31 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:31 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-132", "mode": "writeback"}]': finished 2024-09-16T08:59:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:31 smithi169 ceph-mon[33465]: osdmap e647: 8 total, 8 up, 8 in 2024-09-16T08:59:32.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:32 smithi160 ceph-mon[30137]: pgmap v989: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:59:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:32 smithi160 ceph-mon[40028]: pgmap v989: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:59:32.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:32 smithi169 ceph-mon[33465]: pgmap v989: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:59:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:33 smithi160 ceph-mon[30137]: osdmap e648: 8 total, 8 up, 8 in 2024-09-16T08:59:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:33 smithi160 ceph-mon[40028]: osdmap e648: 8 total, 8 up, 8 in 2024-09-16T08:59:33.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:33 smithi169 ceph-mon[33465]: osdmap e648: 8 total, 8 up, 8 in 2024-09-16T08:59:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:34 smithi160 ceph-mon[30137]: pgmap v992: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:34 smithi160 ceph-mon[30137]: osdmap e649: 8 total, 8 up, 8 in 2024-09-16T08:59:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:34 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:59:34.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:34 smithi160 ceph-mon[40028]: pgmap v992: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:34.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:34 smithi160 ceph-mon[40028]: osdmap e649: 8 total, 8 up, 8 in 2024-09-16T08:59:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:34 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:59:34.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:34.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:34 smithi169 ceph-mon[33465]: pgmap v992: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:34 smithi169 ceph-mon[33465]: osdmap e649: 8 total, 8 up, 8 in 2024-09-16T08:59:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:34 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:59:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:35.428 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:59:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:59:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:59:35.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:35 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:35 smithi160 ceph-mon[30137]: osdmap e650: 8 total, 8 up, 8 in 2024-09-16T08:59:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:35.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:35 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:35 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:35 smithi160 ceph-mon[40028]: osdmap e650: 8 total, 8 up, 8 in 2024-09-16T08:59:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:35.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:35 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:35.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:59:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:59:35.852 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:35 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:35.853 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:35 smithi169 ceph-mon[33465]: osdmap e650: 8 total, 8 up, 8 in 2024-09-16T08:59:35.853 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:35.853 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:35 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:36 smithi160 ceph-mon[30137]: pgmap v995: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:59:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:36 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"}]': finished 2024-09-16T08:59:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:36 smithi160 ceph-mon[30137]: osdmap e651: 8 total, 8 up, 8 in 2024-09-16T08:59:36.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:36 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:36 smithi160 ceph-mon[40028]: pgmap v995: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:59:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:36 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"}]': finished 2024-09-16T08:59:36.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:36 smithi160 ceph-mon[40028]: osdmap e651: 8 total, 8 up, 8 in 2024-09-16T08:59:36.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:36 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:36.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:36 smithi169 ceph-mon[33465]: pgmap v995: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:59:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:36 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-132"}]': finished 2024-09-16T08:59:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:36 smithi169 ceph-mon[33465]: osdmap e651: 8 total, 8 up, 8 in 2024-09-16T08:59:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:36 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:37 smithi160 ceph-mon[30137]: osdmap e652: 8 total, 8 up, 8 in 2024-09-16T08:59:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:37.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:37 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:37 smithi160 ceph-mon[40028]: osdmap e652: 8 total, 8 up, 8 in 2024-09-16T08:59:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:37.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:37 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:37.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:37 smithi169 ceph-mon[33465]: osdmap e652: 8 total, 8 up, 8 in 2024-09-16T08:59:37.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:37.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:37 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:38 smithi169 ceph-mon[33465]: pgmap v998: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:59:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:38 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:38 smithi169 ceph-mon[33465]: osdmap e653: 8 total, 8 up, 8 in 2024-09-16T08:59:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:38 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:38.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:38 smithi160 ceph-mon[30137]: pgmap v998: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:59:38.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:38 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:38.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:38 smithi160 ceph-mon[30137]: osdmap e653: 8 total, 8 up, 8 in 2024-09-16T08:59:38.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:38.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:38 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:38.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:38 smithi160 ceph-mon[40028]: pgmap v998: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:59:38.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:38 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:38.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:38 smithi160 ceph-mon[40028]: osdmap e653: 8 total, 8 up, 8 in 2024-09-16T08:59:38.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:38.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:38 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132"} : dispatch 2024-09-16T08:59:39.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:39 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:39.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:39 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132"}]': finished 2024-09-16T08:59:39.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:39 smithi169 ceph-mon[33465]: osdmap e654: 8 total, 8 up, 8 in 2024-09-16T08:59:39.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:39 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:39.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:39 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132"}]': finished 2024-09-16T08:59:39.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:39 smithi160 ceph-mon[30137]: osdmap e654: 8 total, 8 up, 8 in 2024-09-16T08:59:39.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:39 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:39.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:39 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-132"}]': finished 2024-09-16T08:59:39.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:39 smithi160 ceph-mon[40028]: osdmap e654: 8 total, 8 up, 8 in 2024-09-16T08:59:40.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:40 smithi169 ceph-mon[33465]: pgmap v1001: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-16T08:59:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:40 smithi169 ceph-mon[33465]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:59:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:40 smithi169 ceph-mon[33465]: osdmap e655: 8 total, 8 up, 8 in 2024-09-16T08:59:40.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:40 smithi160 ceph-mon[30137]: pgmap v1001: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-16T08:59:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:40 smithi160 ceph-mon[30137]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:59:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:40 smithi160 ceph-mon[30137]: osdmap e655: 8 total, 8 up, 8 in 2024-09-16T08:59:40.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:40 smithi160 ceph-mon[40028]: pgmap v1001: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-16T08:59:40.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:40 smithi160 ceph-mon[40028]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:59:40.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:40 smithi160 ceph-mon[40028]: osdmap e655: 8 total, 8 up, 8 in 2024-09-16T08:59:41.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:41 smithi169 ceph-mon[33465]: osdmap e656: 8 total, 8 up, 8 in 2024-09-16T08:59:41.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:41.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:41 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:41.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:41 smithi160 ceph-mon[30137]: osdmap e656: 8 total, 8 up, 8 in 2024-09-16T08:59:41.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:41.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:41 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:41.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:41 smithi160 ceph-mon[40028]: osdmap e656: 8 total, 8 up, 8 in 2024-09-16T08:59:41.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:41.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:41 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:42 smithi169 ceph-mon[33465]: pgmap v1004: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:59:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:42 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:42 smithi169 ceph-mon[33465]: osdmap e657: 8 total, 8 up, 8 in 2024-09-16T08:59:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:42 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:42.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:42 smithi160 ceph-mon[30137]: pgmap v1004: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:59:42.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:42 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:42.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:42 smithi160 ceph-mon[30137]: osdmap e657: 8 total, 8 up, 8 in 2024-09-16T08:59:42.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:42.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:42 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:42.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:42 smithi160 ceph-mon[40028]: pgmap v1004: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:59:42.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:42 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:42.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:42 smithi160 ceph-mon[40028]: osdmap e657: 8 total, 8 up, 8 in 2024-09-16T08:59:42.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:42.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:42 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:43 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:43 smithi169 ceph-mon[33465]: osdmap e658: 8 total, 8 up, 8 in 2024-09-16T08:59:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-134"} : dispatch 2024-09-16T08:59:43.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:43 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-134"} : dispatch 2024-09-16T08:59:43.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:43 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:43.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:43 smithi160 ceph-mon[30137]: osdmap e658: 8 total, 8 up, 8 in 2024-09-16T08:59:43.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-134"} : dispatch 2024-09-16T08:59:43.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:43 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-134"} : dispatch 2024-09-16T08:59:43.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:43 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:43.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:43 smithi160 ceph-mon[40028]: osdmap e658: 8 total, 8 up, 8 in 2024-09-16T08:59:43.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-134"} : dispatch 2024-09-16T08:59:43.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:43 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-134"} : dispatch 2024-09-16T08:59:44.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:44 smithi169 ceph-mon[33465]: pgmap v1007: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:44.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:44 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-134"}]': finished 2024-09-16T08:59:44.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:44 smithi169 ceph-mon[33465]: osdmap e659: 8 total, 8 up, 8 in 2024-09-16T08:59:44.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-134", "mode": "writeback"} : dispatch 2024-09-16T08:59:44.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:44 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-134", "mode": "writeback"} : dispatch 2024-09-16T08:59:44.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:44 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:44.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:44 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-134", "mode": "writeback"}]': finished 2024-09-16T08:59:44.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:44 smithi169 ceph-mon[33465]: osdmap e660: 8 total, 8 up, 8 in 2024-09-16T08:59:44.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:44.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:44 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[40028]: pgmap v1007: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-134"}]': finished 2024-09-16T08:59:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[40028]: osdmap e659: 8 total, 8 up, 8 in 2024-09-16T08:59:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-134", "mode": "writeback"} : dispatch 2024-09-16T08:59:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-134", "mode": "writeback"} : dispatch 2024-09-16T08:59:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-134", "mode": "writeback"}]': finished 2024-09-16T08:59:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[40028]: osdmap e660: 8 total, 8 up, 8 in 2024-09-16T08:59:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:44.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[30137]: pgmap v1007: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:44.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-134"}]': finished 2024-09-16T08:59:44.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[30137]: osdmap e659: 8 total, 8 up, 8 in 2024-09-16T08:59:44.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-134", "mode": "writeback"} : dispatch 2024-09-16T08:59:44.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-134", "mode": "writeback"} : dispatch 2024-09-16T08:59:44.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:44.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-134", "mode": "writeback"}]': finished 2024-09-16T08:59:44.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[30137]: osdmap e660: 8 total, 8 up, 8 in 2024-09-16T08:59:44.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:44.950 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:45.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:59:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:59:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:59:45.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:59:45 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:59:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:46 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:46 smithi169 ceph-mon[33465]: osdmap e661: 8 total, 8 up, 8 in 2024-09-16T08:59:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:46 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134"} : dispatch 2024-09-16T08:59:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:46 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134"} : dispatch 2024-09-16T08:59:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:46 smithi169 ceph-mon[33465]: pgmap v1011: 204 pgs: 204 active+clean; 583 KiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:59:46.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:46 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:46 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:46 smithi160 ceph-mon[30137]: osdmap e661: 8 total, 8 up, 8 in 2024-09-16T08:59:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:46 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134"} : dispatch 2024-09-16T08:59:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:46 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134"} : dispatch 2024-09-16T08:59:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:46 smithi160 ceph-mon[30137]: pgmap v1011: 204 pgs: 204 active+clean; 583 KiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:59:46.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:46 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:46 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:46.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:46 smithi160 ceph-mon[40028]: osdmap e661: 8 total, 8 up, 8 in 2024-09-16T08:59:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:46 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134"} : dispatch 2024-09-16T08:59:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:46 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134"} : dispatch 2024-09-16T08:59:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:46 smithi160 ceph-mon[40028]: pgmap v1011: 204 pgs: 204 active+clean; 583 KiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:59:46.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:46 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:47 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:47 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134"}]': finished 2024-09-16T08:59:47.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:47 smithi169 ceph-mon[33465]: osdmap e662: 8 total, 8 up, 8 in 2024-09-16T08:59:47.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:47 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:47.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:47 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134"}]': finished 2024-09-16T08:59:47.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:47 smithi160 ceph-mon[30137]: osdmap e662: 8 total, 8 up, 8 in 2024-09-16T08:59:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:47 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:47 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-134"}]': finished 2024-09-16T08:59:47.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:47 smithi160 ceph-mon[40028]: osdmap e662: 8 total, 8 up, 8 in 2024-09-16T08:59:48.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:48 smithi169 ceph-mon[33465]: osdmap e663: 8 total, 8 up, 8 in 2024-09-16T08:59:48.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:48 smithi169 ceph-mon[33465]: pgmap v1014: 172 pgs: 172 active+clean; 583 KiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:59:48.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:48 smithi160 ceph-mon[30137]: osdmap e663: 8 total, 8 up, 8 in 2024-09-16T08:59:48.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:48 smithi160 ceph-mon[30137]: pgmap v1014: 172 pgs: 172 active+clean; 583 KiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:59:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:48 smithi160 ceph-mon[40028]: osdmap e663: 8 total, 8 up, 8 in 2024-09-16T08:59:48.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:48 smithi160 ceph-mon[40028]: pgmap v1014: 172 pgs: 172 active+clean; 583 KiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:59:49.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:49 smithi169 ceph-mon[33465]: osdmap e664: 8 total, 8 up, 8 in 2024-09-16T08:59:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:49 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:59:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:49.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:49 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:59:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:49 smithi160 ceph-mon[30137]: osdmap e664: 8 total, 8 up, 8 in 2024-09-16T08:59:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:49 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:59:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:49.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:49 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:59:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:49 smithi160 ceph-mon[40028]: osdmap e664: 8 total, 8 up, 8 in 2024-09-16T08:59:49.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:49 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T08:59:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:49.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:49 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:59:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:50 smithi169 ceph-mon[33465]: pgmap v1016: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:59:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:50 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:50 smithi169 ceph-mon[33465]: osdmap e665: 8 total, 8 up, 8 in 2024-09-16T08:59:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:50 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:50 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:50 smithi169 ceph-mon[33465]: osdmap e666: 8 total, 8 up, 8 in 2024-09-16T08:59:50.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:50 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-136"} : dispatch 2024-09-16T08:59:50.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:50 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-136"} : dispatch 2024-09-16T08:59:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[30137]: pgmap v1016: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:59:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[30137]: osdmap e665: 8 total, 8 up, 8 in 2024-09-16T08:59:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[30137]: osdmap e666: 8 total, 8 up, 8 in 2024-09-16T08:59:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-136"} : dispatch 2024-09-16T08:59:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-136"} : dispatch 2024-09-16T08:59:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[40028]: pgmap v1016: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:59:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[40028]: osdmap e665: 8 total, 8 up, 8 in 2024-09-16T08:59:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[40028]: osdmap e666: 8 total, 8 up, 8 in 2024-09-16T08:59:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-136"} : dispatch 2024-09-16T08:59:50.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:50 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-136"} : dispatch 2024-09-16T08:59:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:52 smithi169 ceph-mon[33465]: pgmap v1019: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:59:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:52 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-136"}]': finished 2024-09-16T08:59:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:52 smithi169 ceph-mon[33465]: osdmap e667: 8 total, 8 up, 8 in 2024-09-16T08:59:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-136", "mode": "writeback"} : dispatch 2024-09-16T08:59:52.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:52 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-136", "mode": "writeback"} : dispatch 2024-09-16T08:59:52.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:52 smithi160 ceph-mon[30137]: pgmap v1019: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:59:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:52 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-136"}]': finished 2024-09-16T08:59:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:52 smithi160 ceph-mon[30137]: osdmap e667: 8 total, 8 up, 8 in 2024-09-16T08:59:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-136", "mode": "writeback"} : dispatch 2024-09-16T08:59:52.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:52 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-136", "mode": "writeback"} : dispatch 2024-09-16T08:59:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:52 smithi160 ceph-mon[40028]: pgmap v1019: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:59:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:52 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-136"}]': finished 2024-09-16T08:59:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:52 smithi160 ceph-mon[40028]: osdmap e667: 8 total, 8 up, 8 in 2024-09-16T08:59:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-136", "mode": "writeback"} : dispatch 2024-09-16T08:59:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:52 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-136", "mode": "writeback"} : dispatch 2024-09-16T08:59:53.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:53 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:53.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:53 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-136", "mode": "writeback"}]': finished 2024-09-16T08:59:53.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:53 smithi169 ceph-mon[33465]: osdmap e668: 8 total, 8 up, 8 in 2024-09-16T08:59:53.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:53.601 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:53 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:53.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:53 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:53 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-136", "mode": "writeback"}]': finished 2024-09-16T08:59:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:53 smithi160 ceph-mon[30137]: osdmap e668: 8 total, 8 up, 8 in 2024-09-16T08:59:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:53.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:53 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:53 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T08:59:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:53 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-136", "mode": "writeback"}]': finished 2024-09-16T08:59:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:53 smithi160 ceph-mon[40028]: osdmap e668: 8 total, 8 up, 8 in 2024-09-16T08:59:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:53.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:53 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T08:59:54.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:54 smithi169 ceph-mon[33465]: pgmap v1022: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 671 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:54 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:54 smithi169 ceph-mon[33465]: osdmap e669: 8 total, 8 up, 8 in 2024-09-16T08:59:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136"} : dispatch 2024-09-16T08:59:54.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:54 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136"} : dispatch 2024-09-16T08:59:54.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:54 smithi160 ceph-mon[30137]: pgmap v1022: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 671 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:54.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:54 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:54.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:54 smithi160 ceph-mon[30137]: osdmap e669: 8 total, 8 up, 8 in 2024-09-16T08:59:54.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136"} : dispatch 2024-09-16T08:59:54.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:54 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136"} : dispatch 2024-09-16T08:59:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:54 smithi160 ceph-mon[40028]: pgmap v1022: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 671 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:59:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:54 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T08:59:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:54 smithi160 ceph-mon[40028]: osdmap e669: 8 total, 8 up, 8 in 2024-09-16T08:59:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136"} : dispatch 2024-09-16T08:59:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:54 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136"} : dispatch 2024-09-16T08:59:55.343 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 08:59:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:08:59:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T08:59:55.599 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:55 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:55.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:55 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136"}]': finished 2024-09-16T08:59:55.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:55 smithi169 ceph-mon[33465]: osdmap e670: 8 total, 8 up, 8 in 2024-09-16T08:59:55.600 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 08:59:55 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T08:59:55.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:55 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:55.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:55 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136"}]': finished 2024-09-16T08:59:55.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:55 smithi160 ceph-mon[30137]: osdmap e670: 8 total, 8 up, 8 in 2024-09-16T08:59:55.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:55 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T08:59:55.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:55 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-136"}]': finished 2024-09-16T08:59:55.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:55 smithi160 ceph-mon[40028]: osdmap e670: 8 total, 8 up, 8 in 2024-09-16T08:59:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:56 smithi169 ceph-mon[33465]: pgmap v1025: 204 pgs: 204 active+clean; 4.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-09-16T08:59:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:56 smithi169 ceph-mon[33465]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:59:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:56 smithi169 ceph-mon[33465]: osdmap e671: 8 total, 8 up, 8 in 2024-09-16T08:59:56.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:56 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:56 smithi160 ceph-mon[30137]: pgmap v1025: 204 pgs: 204 active+clean; 4.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-09-16T08:59:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:56 smithi160 ceph-mon[30137]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:59:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:56 smithi160 ceph-mon[30137]: osdmap e671: 8 total, 8 up, 8 in 2024-09-16T08:59:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:56 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:56 smithi160 ceph-mon[40028]: pgmap v1025: 204 pgs: 204 active+clean; 4.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-09-16T08:59:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:56 smithi160 ceph-mon[40028]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:59:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:56 smithi160 ceph-mon[40028]: osdmap e671: 8 total, 8 up, 8 in 2024-09-16T08:59:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:56 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:59:57.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:57 smithi160 ceph-mon[30137]: osdmap e672: 8 total, 8 up, 8 in 2024-09-16T08:59:57.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:57.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:57 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:57.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:57 smithi160 ceph-mon[40028]: osdmap e672: 8 total, 8 up, 8 in 2024-09-16T08:59:57.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:57 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:57.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:57 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:57.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:57 smithi169 ceph-mon[33465]: osdmap e672: 8 total, 8 up, 8 in 2024-09-16T08:59:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:57 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:57 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T08:59:58.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:58 smithi160 ceph-mon[40028]: pgmap v1028: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-16T08:59:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:58 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:58 smithi160 ceph-mon[40028]: osdmap e673: 8 total, 8 up, 8 in 2024-09-16T08:59:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:58 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:58.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:58 smithi160 ceph-mon[30137]: pgmap v1028: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-16T08:59:58.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:58 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:58.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:58 smithi160 ceph-mon[30137]: osdmap e673: 8 total, 8 up, 8 in 2024-09-16T08:59:58.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:58.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:58 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:58 smithi169 ceph-mon[33465]: pgmap v1028: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-16T08:59:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:58 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T08:59:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:58 smithi169 ceph-mon[33465]: osdmap e673: 8 total, 8 up, 8 in 2024-09-16T08:59:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:58 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T08:59:59.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:59 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:59 smithi160 ceph-mon[30137]: osdmap e674: 8 total, 8 up, 8 in 2024-09-16T08:59:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:59:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 08:59:59 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:59:59.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:59 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:59.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:59 smithi160 ceph-mon[40028]: osdmap e674: 8 total, 8 up, 8 in 2024-09-16T08:59:59.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:59:59.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 08:59:59 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:59:59.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:59 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T08:59:59.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:59 smithi169 ceph-mon[33465]: osdmap e674: 8 total, 8 up, 8 in 2024-09-16T08:59:59.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T08:59:59.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 08:59:59 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T09:00:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[30137]: pgmap v1031: 204 pgs: 204 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_count","val": "2"}]': finished 2024-09-16T09:00:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[30137]: osdmap e675: 8 total, 8 up, 8 in 2024-09-16T09:00:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:00:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:00:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[30137]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-09-16T09:00:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[30137]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-16T09:00:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[30137]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-16T09:00:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[30137]: application not enabled on pool 'test-rados-api-smithi160-76079-111' 2024-09-16T09:00:00.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[30137]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-16T09:00:00.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[40028]: pgmap v1031: 204 pgs: 204 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_count","val": "2"}]': finished 2024-09-16T09:00:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[40028]: osdmap e675: 8 total, 8 up, 8 in 2024-09-16T09:00:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:00:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:00:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[40028]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-09-16T09:00:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[40028]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-16T09:00:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[40028]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-16T09:00:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[40028]: application not enabled on pool 'test-rados-api-smithi160-76079-111' 2024-09-16T09:00:00.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:00 smithi160 ceph-mon[40028]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-16T09:00:00.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:00 smithi169 ceph-mon[33465]: pgmap v1031: 204 pgs: 204 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:00.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:00 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_count","val": "2"}]': finished 2024-09-16T09:00:00.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:00 smithi169 ceph-mon[33465]: osdmap e675: 8 total, 8 up, 8 in 2024-09-16T09:00:00.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:00:00.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:00 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:00:00.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:00 smithi169 ceph-mon[33465]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-09-16T09:00:00.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:00 smithi169 ceph-mon[33465]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-16T09:00:00.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:00 smithi169 ceph-mon[33465]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-16T09:00:00.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:00 smithi169 ceph-mon[33465]: application not enabled on pool 'test-rados-api-smithi160-76079-111' 2024-09-16T09:00:00.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:00 smithi169 ceph-mon[33465]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-16T09:00:01.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:01 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_period","val": "600"}]': finished 2024-09-16T09:00:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:01 smithi160 ceph-mon[30137]: osdmap e676: 8 total, 8 up, 8 in 2024-09-16T09:00:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-16T09:00:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:01 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-16T09:00:01.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:01 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_period","val": "600"}]': finished 2024-09-16T09:00:01.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:01 smithi160 ceph-mon[40028]: osdmap e676: 8 total, 8 up, 8 in 2024-09-16T09:00:01.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-16T09:00:01.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:01 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-16T09:00:01.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:01 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_period","val": "600"}]': finished 2024-09-16T09:00:01.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:01 smithi169 ceph-mon[33465]: osdmap e676: 8 total, 8 up, 8 in 2024-09-16T09:00:01.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-16T09:00:01.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:01 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-16T09:00:02.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:02 smithi160 ceph-mon[40028]: pgmap v1034: 204 pgs: 204 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:02.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:02 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-16T09:00:02.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:02 smithi160 ceph-mon[40028]: osdmap e677: 8 total, 8 up, 8 in 2024-09-16T09:00:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:02 smithi160 ceph-mon[30137]: pgmap v1034: 204 pgs: 204 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:02 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-16T09:00:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:02 smithi160 ceph-mon[30137]: osdmap e677: 8 total, 8 up, 8 in 2024-09-16T09:00:02.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:02 smithi169 ceph-mon[33465]: pgmap v1034: 204 pgs: 204 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:02.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:02 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-16T09:00:02.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:02 smithi169 ceph-mon[33465]: osdmap e677: 8 total, 8 up, 8 in 2024-09-16T09:00:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:03 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138"} : dispatch 2024-09-16T09:00:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:03 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138"} : dispatch 2024-09-16T09:00:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:03 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138"} : dispatch 2024-09-16T09:00:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:03 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138"} : dispatch 2024-09-16T09:00:03.770 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:03.770 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:03 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:03.770 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138"} : dispatch 2024-09-16T09:00:03.770 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:03 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138"} : dispatch 2024-09-16T09:00:04.697 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:04 smithi160 ceph-mon[40028]: pgmap v1036: 204 pgs: 204 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T09:00:04.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:04 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138"}]': finished 2024-09-16T09:00:04.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:04 smithi160 ceph-mon[40028]: osdmap e678: 8 total, 8 up, 8 in 2024-09-16T09:00:04.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:04.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:04 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:00:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:04 smithi160 ceph-mon[30137]: pgmap v1036: 204 pgs: 204 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T09:00:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:04 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138"}]': finished 2024-09-16T09:00:04.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:04 smithi160 ceph-mon[30137]: osdmap e678: 8 total, 8 up, 8 in 2024-09-16T09:00:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:04.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:04 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:00:04.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:04 smithi169 ceph-mon[33465]: pgmap v1036: 204 pgs: 204 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T09:00:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:04 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-138"}]': finished 2024-09-16T09:00:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:04 smithi169 ceph-mon[33465]: osdmap e678: 8 total, 8 up, 8 in 2024-09-16T09:00:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:04 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:00:05.425 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:00:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:00:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:00:05.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:05 smithi160 ceph-mon[30137]: osdmap e679: 8 total, 8 up, 8 in 2024-09-16T09:00:05.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:05 smithi160 ceph-mon[40028]: osdmap e679: 8 total, 8 up, 8 in 2024-09-16T09:00:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:05 smithi169 ceph-mon[33465]: osdmap e679: 8 total, 8 up, 8 in 2024-09-16T09:00:05.850 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:00:05 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:00:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:06 smithi160 ceph-mon[30137]: pgmap v1039: 172 pgs: 172 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T09:00:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:06 smithi160 ceph-mon[30137]: osdmap e680: 8 total, 8 up, 8 in 2024-09-16T09:00:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:06 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:06 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:06.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:06 smithi160 ceph-mon[40028]: pgmap v1039: 172 pgs: 172 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T09:00:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:06 smithi160 ceph-mon[40028]: osdmap e680: 8 total, 8 up, 8 in 2024-09-16T09:00:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:06 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:06 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:06.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:06 smithi169 ceph-mon[33465]: pgmap v1039: 172 pgs: 172 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T09:00:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:06 smithi169 ceph-mon[33465]: osdmap e680: 8 total, 8 up, 8 in 2024-09-16T09:00:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:06 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:06 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:07.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:07 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:00:07.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:07 smithi160 ceph-mon[30137]: osdmap e681: 8 total, 8 up, 8 in 2024-09-16T09:00:07.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:07 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:07.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:07 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:07.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:07 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:00:07.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:07 smithi160 ceph-mon[40028]: osdmap e681: 8 total, 8 up, 8 in 2024-09-16T09:00:07.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:07 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:07.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:07 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:07.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:07 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:00:07.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:07 smithi169 ceph-mon[33465]: osdmap e681: 8 total, 8 up, 8 in 2024-09-16T09:00:07.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:07 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:07.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:07 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:08 smithi169 ceph-mon[33465]: pgmap v1042: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:08 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T09:00:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:08 smithi169 ceph-mon[33465]: osdmap e682: 8 total, 8 up, 8 in 2024-09-16T09:00:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-16T09:00:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:08 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-16T09:00:08.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:08 smithi160 ceph-mon[30137]: pgmap v1042: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:08 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T09:00:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:08 smithi160 ceph-mon[30137]: osdmap e682: 8 total, 8 up, 8 in 2024-09-16T09:00:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-16T09:00:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:08 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-16T09:00:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:08 smithi160 ceph-mon[40028]: pgmap v1042: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:08 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T09:00:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:08 smithi160 ceph-mon[40028]: osdmap e682: 8 total, 8 up, 8 in 2024-09-16T09:00:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-16T09:00:08.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:08 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-16T09:00:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:09 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_count","val": "3"}]': finished 2024-09-16T09:00:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:09 smithi169 ceph-mon[33465]: osdmap e683: 8 total, 8 up, 8 in 2024-09-16T09:00:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:09 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-16T09:00:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:09 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-16T09:00:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:09 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_period","val": "3"}]': finished 2024-09-16T09:00:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:09 smithi169 ceph-mon[33465]: osdmap e684: 8 total, 8 up, 8 in 2024-09-16T09:00:09.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:09 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_count","val": "3"}]': finished 2024-09-16T09:00:09.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:09 smithi160 ceph-mon[30137]: osdmap e683: 8 total, 8 up, 8 in 2024-09-16T09:00:09.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:09 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-16T09:00:09.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:09 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-16T09:00:09.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:09 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_period","val": "3"}]': finished 2024-09-16T09:00:09.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:09 smithi160 ceph-mon[30137]: osdmap e684: 8 total, 8 up, 8 in 2024-09-16T09:00:09.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:09 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_count","val": "3"}]': finished 2024-09-16T09:00:09.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:09 smithi160 ceph-mon[40028]: osdmap e683: 8 total, 8 up, 8 in 2024-09-16T09:00:09.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:09 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-16T09:00:09.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:09 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-16T09:00:09.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:09 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_period","val": "3"}]': finished 2024-09-16T09:00:09.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:09 smithi160 ceph-mon[40028]: osdmap e684: 8 total, 8 up, 8 in 2024-09-16T09:00:10.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:10 smithi169 ceph-mon[33465]: pgmap v1045: 204 pgs: 204 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:10 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:00:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:10 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:00:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:10 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T09:00:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:10 smithi169 ceph-mon[33465]: osdmap e685: 8 total, 8 up, 8 in 2024-09-16T09:00:10.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:10 smithi160 ceph-mon[30137]: pgmap v1045: 204 pgs: 204 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:10.993 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:10 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:00:10.993 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:10 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:00:10.994 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:10 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T09:00:10.994 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:10 smithi160 ceph-mon[30137]: osdmap e685: 8 total, 8 up, 8 in 2024-09-16T09:00:10.994 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:10 smithi160 ceph-mon[40028]: pgmap v1045: 204 pgs: 204 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:10.994 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:10 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:00:10.994 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:10 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:00:10.995 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:10 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T09:00:10.995 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:10 smithi160 ceph-mon[40028]: osdmap e685: 8 total, 8 up, 8 in 2024-09-16T09:00:11.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:11 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-16T09:00:11.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:11 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-16T09:00:11.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:11 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-16T09:00:11.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:11 smithi169 ceph-mon[33465]: osdmap e686: 8 total, 8 up, 8 in 2024-09-16T09:00:11.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:11 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-16T09:00:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:11 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-16T09:00:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:11 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-16T09:00:11.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:11 smithi160 ceph-mon[30137]: osdmap e686: 8 total, 8 up, 8 in 2024-09-16T09:00:11.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:11 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-16T09:00:11.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:11 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-16T09:00:11.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:11 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-16T09:00:11.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:11 smithi160 ceph-mon[40028]: osdmap e686: 8 total, 8 up, 8 in 2024-09-16T09:00:12.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:12 smithi169 ceph-mon[33465]: pgmap v1048: 204 pgs: 204 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:12.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:12 smithi160 ceph-mon[30137]: pgmap v1048: 204 pgs: 204 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:12.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:12 smithi160 ceph-mon[40028]: pgmap v1048: 204 pgs: 204 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:14.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:14 smithi169 ceph-mon[33465]: pgmap v1050: 204 pgs: 204 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T09:00:14.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:14 smithi160 ceph-mon[30137]: pgmap v1050: 204 pgs: 204 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T09:00:14.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:14 smithi160 ceph-mon[40028]: pgmap v1050: 204 pgs: 204 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T09:00:15.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:00:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:00:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:00:15.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:00:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:00:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:16 smithi169 ceph-mon[33465]: pgmap v1051: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-16T09:00:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:16 smithi160 ceph-mon[30137]: pgmap v1051: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-16T09:00:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:16.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:16 smithi160 ceph-mon[40028]: pgmap v1051: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-16T09:00:16.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:18.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:18 smithi169 ceph-mon[33465]: pgmap v1052: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 651 B/s rd, 3.1 KiB/s wr, 1 op/s 2024-09-16T09:00:18.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:18 smithi160 ceph-mon[30137]: pgmap v1052: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 651 B/s rd, 3.1 KiB/s wr, 1 op/s 2024-09-16T09:00:18.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:18 smithi160 ceph-mon[40028]: pgmap v1052: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 651 B/s rd, 3.1 KiB/s wr, 1 op/s 2024-09-16T09:00:19.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:19.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:19.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:20.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:20 smithi169 ceph-mon[33465]: pgmap v1053: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 7.4 KiB/s wr, 2 op/s 2024-09-16T09:00:20.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:20 smithi160 ceph-mon[30137]: pgmap v1053: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 7.4 KiB/s wr, 2 op/s 2024-09-16T09:00:20.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:20 smithi160 ceph-mon[40028]: pgmap v1053: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 7.4 KiB/s wr, 2 op/s 2024-09-16T09:00:22.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:22 smithi169 ceph-mon[33465]: pgmap v1054: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-09-16T09:00:22.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:22 smithi160 ceph-mon[30137]: pgmap v1054: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-09-16T09:00:22.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:22 smithi160 ceph-mon[40028]: pgmap v1054: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-09-16T09:00:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:24 smithi169 ceph-mon[33465]: pgmap v1055: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 863 B/s rd, 5.6 KiB/s wr, 1 op/s 2024-09-16T09:00:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:24 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140"} : dispatch 2024-09-16T09:00:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:24 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140"} : dispatch 2024-09-16T09:00:24.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:24 smithi160 ceph-mon[30137]: pgmap v1055: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 863 B/s rd, 5.6 KiB/s wr, 1 op/s 2024-09-16T09:00:24.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:24.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:24 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:24.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140"} : dispatch 2024-09-16T09:00:24.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:24 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140"} : dispatch 2024-09-16T09:00:24.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:24 smithi160 ceph-mon[40028]: pgmap v1055: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 863 B/s rd, 5.6 KiB/s wr, 1 op/s 2024-09-16T09:00:24.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:24.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:24 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:24.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140"} : dispatch 2024-09-16T09:00:24.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:24 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140"} : dispatch 2024-09-16T09:00:25.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:00:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:00:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:00:25.850 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:00:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:00:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:25 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140"}]': finished 2024-09-16T09:00:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:25 smithi169 ceph-mon[33465]: osdmap e687: 8 total, 8 up, 8 in 2024-09-16T09:00:25.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:25 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140"}]': finished 2024-09-16T09:00:25.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:25 smithi160 ceph-mon[30137]: osdmap e687: 8 total, 8 up, 8 in 2024-09-16T09:00:25.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:25 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-140"}]': finished 2024-09-16T09:00:25.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:25 smithi160 ceph-mon[40028]: osdmap e687: 8 total, 8 up, 8 in 2024-09-16T09:00:26.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:26 smithi169 ceph-mon[33465]: pgmap v1057: 204 pgs: 204 active+clean; 4.5 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 8.4 KiB/s wr, 2 op/s 2024-09-16T09:00:26.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:26.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:26 smithi169 ceph-mon[33465]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:00:26.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:26 smithi169 ceph-mon[33465]: osdmap e688: 8 total, 8 up, 8 in 2024-09-16T09:00:26.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:26 smithi160 ceph-mon[30137]: pgmap v1057: 204 pgs: 204 active+clean; 4.5 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 8.4 KiB/s wr, 2 op/s 2024-09-16T09:00:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:26 smithi160 ceph-mon[30137]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:00:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:26 smithi160 ceph-mon[30137]: osdmap e688: 8 total, 8 up, 8 in 2024-09-16T09:00:26.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:26 smithi160 ceph-mon[40028]: pgmap v1057: 204 pgs: 204 active+clean; 4.5 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 8.4 KiB/s wr, 2 op/s 2024-09-16T09:00:26.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:26.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:26 smithi160 ceph-mon[40028]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:00:26.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:26 smithi160 ceph-mon[40028]: osdmap e688: 8 total, 8 up, 8 in 2024-09-16T09:00:27.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:27 smithi169 ceph-mon[33465]: osdmap e689: 8 total, 8 up, 8 in 2024-09-16T09:00:27.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:27.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:27 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:27.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:27 smithi160 ceph-mon[30137]: osdmap e689: 8 total, 8 up, 8 in 2024-09-16T09:00:27.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:27.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:27 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:27.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:27 smithi160 ceph-mon[40028]: osdmap e689: 8 total, 8 up, 8 in 2024-09-16T09:00:27.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:27.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:27 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:28 smithi169 ceph-mon[33465]: pgmap v1060: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:00:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:28 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:00:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:28 smithi169 ceph-mon[33465]: osdmap e690: 8 total, 8 up, 8 in 2024-09-16T09:00:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:28 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:28 smithi160 ceph-mon[30137]: pgmap v1060: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:00:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:28 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:00:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:28 smithi160 ceph-mon[30137]: osdmap e690: 8 total, 8 up, 8 in 2024-09-16T09:00:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:28 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:28.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:28 smithi160 ceph-mon[40028]: pgmap v1060: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:00:28.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:28 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:00:28.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:28 smithi160 ceph-mon[40028]: osdmap e690: 8 total, 8 up, 8 in 2024-09-16T09:00:28.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:28.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:28 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:29.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:29 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T09:00:29.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:29 smithi169 ceph-mon[33465]: osdmap e691: 8 total, 8 up, 8 in 2024-09-16T09:00:29.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:29.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:29 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:29.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:29 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T09:00:29.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:29 smithi160 ceph-mon[30137]: osdmap e691: 8 total, 8 up, 8 in 2024-09-16T09:00:29.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:29.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:29 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:29.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:29 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T09:00:29.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:29 smithi160 ceph-mon[40028]: osdmap e691: 8 total, 8 up, 8 in 2024-09-16T09:00:29.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:29.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:29 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:30.848 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:30 smithi160 ceph-mon[30137]: pgmap v1063: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:00:30.849 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:30 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-142"}]': finished 2024-09-16T09:00:30.849 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:30 smithi160 ceph-mon[30137]: osdmap e692: 8 total, 8 up, 8 in 2024-09-16T09:00:30.849 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-142", "mode": "writeback"} : dispatch 2024-09-16T09:00:30.849 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:30 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-142", "mode": "writeback"} : dispatch 2024-09-16T09:00:30.849 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:30 smithi160 ceph-mon[40028]: pgmap v1063: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:00:30.849 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:30 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-142"}]': finished 2024-09-16T09:00:30.849 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:30 smithi160 ceph-mon[40028]: osdmap e692: 8 total, 8 up, 8 in 2024-09-16T09:00:30.849 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-142", "mode": "writeback"} : dispatch 2024-09-16T09:00:30.850 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:30 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-142", "mode": "writeback"} : dispatch 2024-09-16T09:00:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:30 smithi169 ceph-mon[33465]: pgmap v1063: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:00:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:30 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-142"}]': finished 2024-09-16T09:00:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:30 smithi169 ceph-mon[33465]: osdmap e692: 8 total, 8 up, 8 in 2024-09-16T09:00:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-142", "mode": "writeback"} : dispatch 2024-09-16T09:00:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:30 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-142", "mode": "writeback"} : dispatch 2024-09-16T09:00:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:31 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T09:00:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:31 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-142", "mode": "writeback"}]': finished 2024-09-16T09:00:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:31 smithi169 ceph-mon[33465]: osdmap e693: 8 total, 8 up, 8 in 2024-09-16T09:00:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T09:00:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:31 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T09:00:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:31 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:00:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:31 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:31 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:00:31.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:31 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:00:31.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:31 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_count","val": "2"}]': finished 2024-09-16T09:00:31.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:31 smithi169 ceph-mon[33465]: osdmap e694: 8 total, 8 up, 8 in 2024-09-16T09:00:31.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:00:31.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:31 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:00:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T09:00:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-142", "mode": "writeback"}]': finished 2024-09-16T09:00:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[30137]: osdmap e693: 8 total, 8 up, 8 in 2024-09-16T09:00:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T09:00:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T09:00:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:00:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:00:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:00:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_count","val": "2"}]': finished 2024-09-16T09:00:31.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[30137]: osdmap e694: 8 total, 8 up, 8 in 2024-09-16T09:00:31.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:00:31.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:00:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T09:00:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-142", "mode": "writeback"}]': finished 2024-09-16T09:00:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[40028]: osdmap e693: 8 total, 8 up, 8 in 2024-09-16T09:00:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T09:00:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T09:00:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:00:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:31.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:00:31.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:00:31.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_count","val": "2"}]': finished 2024-09-16T09:00:31.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[40028]: osdmap e694: 8 total, 8 up, 8 in 2024-09-16T09:00:31.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:00:31.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:31 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:00:32.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:32 smithi169 ceph-mon[33465]: pgmap v1066: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-16T09:00:32.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:32 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_period","val": "600"}]': finished 2024-09-16T09:00:32.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:32 smithi169 ceph-mon[33465]: osdmap e695: 8 total, 8 up, 8 in 2024-09-16T09:00:32.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:32 smithi160 ceph-mon[30137]: pgmap v1066: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-16T09:00:32.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:32 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_period","val": "600"}]': finished 2024-09-16T09:00:32.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:32 smithi160 ceph-mon[30137]: osdmap e695: 8 total, 8 up, 8 in 2024-09-16T09:00:32.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:32 smithi160 ceph-mon[40028]: pgmap v1066: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-16T09:00:32.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:32 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_period","val": "600"}]': finished 2024-09-16T09:00:32.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:32 smithi160 ceph-mon[40028]: osdmap e695: 8 total, 8 up, 8 in 2024-09-16T09:00:33.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:00:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:33 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:00:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:33 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T09:00:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:33 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T09:00:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:33 smithi169 ceph-mon[33465]: osdmap e696: 8 total, 8 up, 8 in 2024-09-16T09:00:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:33 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T09:00:33.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:00:33.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:33 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:00:33.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:33 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T09:00:33.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:33 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T09:00:33.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:33 smithi160 ceph-mon[30137]: osdmap e696: 8 total, 8 up, 8 in 2024-09-16T09:00:33.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:33 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T09:00:33.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:00:33.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:33 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:00:33.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:33 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T09:00:33.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:33 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T09:00:33.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:33 smithi160 ceph-mon[40028]: osdmap e696: 8 total, 8 up, 8 in 2024-09-16T09:00:33.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:33 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T09:00:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:34 smithi169 ceph-mon[33465]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:00:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:34 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T09:00:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:34 smithi169 ceph-mon[33465]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:00:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:34 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-16T09:00:34.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:34 smithi160 ceph-mon[30137]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:00:34.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:34 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T09:00:34.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:34.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:34 smithi160 ceph-mon[30137]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:00:34.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:34 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-16T09:00:34.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:34 smithi160 ceph-mon[40028]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 710 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:00:34.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:34 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T09:00:34.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:34.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:34 smithi160 ceph-mon[40028]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:00:34.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:34 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-16T09:00:35.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:00:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:00:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:00:35.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:00:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:00:35.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:35 smithi169 ceph-mon[33465]: osdmap e697: 8 total, 8 up, 8 in 2024-09-16T09:00:35.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-16T09:00:35.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:35 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-16T09:00:35.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:35 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-16T09:00:35.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:35 smithi169 ceph-mon[33465]: osdmap e698: 8 total, 8 up, 8 in 2024-09-16T09:00:35.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:35 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-16T09:00:35.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:35 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-16T09:00:35.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:35 smithi160 ceph-mon[30137]: osdmap e697: 8 total, 8 up, 8 in 2024-09-16T09:00:35.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-16T09:00:35.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:35 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-16T09:00:35.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:35 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-16T09:00:35.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:35 smithi160 ceph-mon[30137]: osdmap e698: 8 total, 8 up, 8 in 2024-09-16T09:00:35.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:35 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-16T09:00:35.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:35 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-16T09:00:35.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:35 smithi160 ceph-mon[40028]: osdmap e697: 8 total, 8 up, 8 in 2024-09-16T09:00:35.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-16T09:00:35.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:35 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-16T09:00:35.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:35 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-16T09:00:35.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:35 smithi160 ceph-mon[40028]: osdmap e698: 8 total, 8 up, 8 in 2024-09-16T09:00:35.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:35 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-16T09:00:35.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:35 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-16T09:00:36.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:36 smithi169 ceph-mon[33465]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 711 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:36 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:36 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-16T09:00:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:36 smithi169 ceph-mon[33465]: osdmap e699: 8 total, 8 up, 8 in 2024-09-16T09:00:36.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:36 smithi160 ceph-mon[30137]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 711 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:36.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:36 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:36.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:36 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-16T09:00:36.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:36 smithi160 ceph-mon[30137]: osdmap e699: 8 total, 8 up, 8 in 2024-09-16T09:00:36.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:36 smithi160 ceph-mon[40028]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 711 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:36.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:36 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:36.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:36 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-16T09:00:36.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:36 smithi160 ceph-mon[40028]: osdmap e699: 8 total, 8 up, 8 in 2024-09-16T09:00:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:38 smithi169 ceph-mon[33465]: pgmap v1075: 204 pgs: 204 active+clean; 4.4 MiB data, 711 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:38 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:38.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:38 smithi160 ceph-mon[40028]: pgmap v1075: 204 pgs: 204 active+clean; 4.4 MiB data, 711 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:38.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:38.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:38 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:38.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:38 smithi160 ceph-mon[30137]: pgmap v1075: 204 pgs: 204 active+clean; 4.4 MiB data, 711 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:00:38.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:38.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:38 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:39.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:39 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T09:00:39.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:39.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:39 smithi169 ceph-mon[33465]: osdmap e700: 8 total, 8 up, 8 in 2024-09-16T09:00:39.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:39 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:39.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:39 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T09:00:39.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:39.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:39 smithi160 ceph-mon[30137]: osdmap e700: 8 total, 8 up, 8 in 2024-09-16T09:00:39.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:39 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:39.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:39 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T09:00:39.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:39.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:39 smithi160 ceph-mon[40028]: osdmap e700: 8 total, 8 up, 8 in 2024-09-16T09:00:39.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:39 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:40.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:40 smithi169 ceph-mon[33465]: pgmap v1077: 204 pgs: 204 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:00:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:40 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142"}]': finished 2024-09-16T09:00:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:40 smithi169 ceph-mon[33465]: osdmap e701: 8 total, 8 up, 8 in 2024-09-16T09:00:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:40 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:40 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:40.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:40 smithi160 ceph-mon[30137]: pgmap v1077: 204 pgs: 204 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:00:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:40 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142"}]': finished 2024-09-16T09:00:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:40 smithi160 ceph-mon[30137]: osdmap e701: 8 total, 8 up, 8 in 2024-09-16T09:00:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:40 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:40 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:40.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:40 smithi160 ceph-mon[40028]: pgmap v1077: 204 pgs: 204 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:00:40.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:40 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142"}]': finished 2024-09-16T09:00:40.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:40 smithi160 ceph-mon[40028]: osdmap e701: 8 total, 8 up, 8 in 2024-09-16T09:00:40.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:40.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:40 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:40.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:40.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:40 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-142"} : dispatch 2024-09-16T09:00:41.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:41 smithi169 ceph-mon[33465]: osdmap e702: 8 total, 8 up, 8 in 2024-09-16T09:00:41.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:41 smithi160 ceph-mon[30137]: osdmap e702: 8 total, 8 up, 8 in 2024-09-16T09:00:41.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:41 smithi160 ceph-mon[40028]: osdmap e702: 8 total, 8 up, 8 in 2024-09-16T09:00:42.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:42 smithi160 ceph-mon[30137]: pgmap v1080: 172 pgs: 172 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T09:00:42.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:42 smithi160 ceph-mon[30137]: osdmap e703: 8 total, 8 up, 8 in 2024-09-16T09:00:42.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:42.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:42 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:42.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:42 smithi160 ceph-mon[40028]: pgmap v1080: 172 pgs: 172 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T09:00:42.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:42 smithi160 ceph-mon[40028]: osdmap e703: 8 total, 8 up, 8 in 2024-09-16T09:00:42.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:42.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:42 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:43.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:42 smithi169 ceph-mon[33465]: pgmap v1080: 172 pgs: 172 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T09:00:43.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:42 smithi169 ceph-mon[33465]: osdmap e703: 8 total, 8 up, 8 in 2024-09-16T09:00:43.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:43.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:42 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:00:43.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:43 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:00:43.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:43 smithi160 ceph-mon[30137]: osdmap e704: 8 total, 8 up, 8 in 2024-09-16T09:00:43.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:43.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:43 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:43.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:43 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T09:00:43.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:43 smithi160 ceph-mon[30137]: osdmap e705: 8 total, 8 up, 8 in 2024-09-16T09:00:43.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:43 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:43.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:43 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:43.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:43 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:00:43.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:43 smithi160 ceph-mon[40028]: osdmap e704: 8 total, 8 up, 8 in 2024-09-16T09:00:43.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:43.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:43 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:43.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:43 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T09:00:43.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:43 smithi160 ceph-mon[40028]: osdmap e705: 8 total, 8 up, 8 in 2024-09-16T09:00:43.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:43 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:43.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:43 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:43 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:00:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:43 smithi169 ceph-mon[33465]: osdmap e704: 8 total, 8 up, 8 in 2024-09-16T09:00:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:43 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:00:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:43 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T09:00:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:43 smithi169 ceph-mon[33465]: osdmap e705: 8 total, 8 up, 8 in 2024-09-16T09:00:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:43 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:43 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:44 smithi160 ceph-mon[30137]: pgmap v1083: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:00:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:44 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-144"}]': finished 2024-09-16T09:00:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:44 smithi160 ceph-mon[30137]: osdmap e706: 8 total, 8 up, 8 in 2024-09-16T09:00:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:44 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-144", "mode": "readproxy"} : dispatch 2024-09-16T09:00:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:44 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-144", "mode": "readproxy"} : dispatch 2024-09-16T09:00:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:44 smithi160 ceph-mon[40028]: pgmap v1083: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:00:44.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:44 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-144"}]': finished 2024-09-16T09:00:44.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:44 smithi160 ceph-mon[40028]: osdmap e706: 8 total, 8 up, 8 in 2024-09-16T09:00:44.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:44 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-144", "mode": "readproxy"} : dispatch 2024-09-16T09:00:44.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:44 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-144", "mode": "readproxy"} : dispatch 2024-09-16T09:00:45.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:44 smithi169 ceph-mon[33465]: pgmap v1083: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:00:45.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:44 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-144"}]': finished 2024-09-16T09:00:45.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:44 smithi169 ceph-mon[33465]: osdmap e706: 8 total, 8 up, 8 in 2024-09-16T09:00:45.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:44 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-144", "mode": "readproxy"} : dispatch 2024-09-16T09:00:45.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:44 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-144", "mode": "readproxy"} : dispatch 2024-09-16T09:00:45.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:00:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:00:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:00:45.850 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:00:45 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:00:45.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:45 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T09:00:45.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:45 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-144", "mode": "readproxy"}]': finished 2024-09-16T09:00:45.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:45 smithi169 ceph-mon[33465]: osdmap e707: 8 total, 8 up, 8 in 2024-09-16T09:00:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:45 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T09:00:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:45 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-144", "mode": "readproxy"}]': finished 2024-09-16T09:00:45.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:45 smithi160 ceph-mon[30137]: osdmap e707: 8 total, 8 up, 8 in 2024-09-16T09:00:45.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:45 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T09:00:45.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:45 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-144", "mode": "readproxy"}]': finished 2024-09-16T09:00:45.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:45 smithi160 ceph-mon[40028]: osdmap e707: 8 total, 8 up, 8 in 2024-09-16T09:00:46.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:46 smithi160 ceph-mon[30137]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T09:00:46.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:46 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:46.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:46 smithi160 ceph-mon[40028]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T09:00:46.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:46 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:47.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:46 smithi169 ceph-mon[33465]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-16T09:00:47.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:46 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:48.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:48 smithi160 ceph-mon[30137]: pgmap v1088: 204 pgs: 204 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 217 B/s wr, 1 op/s 2024-09-16T09:00:48.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:48 smithi160 ceph-mon[40028]: pgmap v1088: 204 pgs: 204 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 217 B/s wr, 1 op/s 2024-09-16T09:00:49.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:48 smithi169 ceph-mon[33465]: pgmap v1088: 204 pgs: 204 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 217 B/s wr, 1 op/s 2024-09-16T09:00:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:50 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:00:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:50 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:50.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:50 smithi169 ceph-mon[33465]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-16T09:00:50.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:50 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:00:50.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:50 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:50.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:50 smithi160 ceph-mon[30137]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-16T09:00:50.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:50 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:00:50.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:50 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:50.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:50 smithi160 ceph-mon[40028]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-16T09:00:52.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:52 smithi160 ceph-mon[30137]: pgmap v1090: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 132 B/s wr, 1 op/s 2024-09-16T09:00:52.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:52 smithi160 ceph-mon[40028]: pgmap v1090: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 132 B/s wr, 1 op/s 2024-09-16T09:00:52.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:52 smithi169 ceph-mon[33465]: pgmap v1090: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 132 B/s wr, 1 op/s 2024-09-16T09:00:54.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:54 smithi160 ceph-mon[30137]: pgmap v1091: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 704 B/s rd, 0 op/s 2024-09-16T09:00:54.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:54 smithi160 ceph-mon[40028]: pgmap v1091: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 704 B/s rd, 0 op/s 2024-09-16T09:00:54.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:54 smithi169 ceph-mon[33465]: pgmap v1091: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 704 B/s rd, 0 op/s 2024-09-16T09:00:55.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:00:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:00:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:00:55.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:00:55 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:00:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:56 smithi160 ceph-mon[40028]: pgmap v1092: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:00:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:56 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:56.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:56 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:56 smithi160 ceph-mon[30137]: pgmap v1092: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:00:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:56 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:56.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:56.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:56 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:56.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:56 smithi169 ceph-mon[33465]: pgmap v1092: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:00:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:56 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:00:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:56 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:57.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:57 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T09:00:57.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:57 smithi160 ceph-mon[30137]: osdmap e708: 8 total, 8 up, 8 in 2024-09-16T09:00:57.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:57.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:57 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:57.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:57 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T09:00:57.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:57 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144"}]': finished 2024-09-16T09:00:57.699 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:57 smithi160 ceph-mon[30137]: osdmap e709: 8 total, 8 up, 8 in 2024-09-16T09:00:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:57 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T09:00:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:57 smithi160 ceph-mon[40028]: osdmap e708: 8 total, 8 up, 8 in 2024-09-16T09:00:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:57 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:57 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:57 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T09:00:57.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:57 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144"}]': finished 2024-09-16T09:00:57.700 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:57 smithi160 ceph-mon[40028]: osdmap e709: 8 total, 8 up, 8 in 2024-09-16T09:00:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:57 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T09:00:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:57 smithi169 ceph-mon[33465]: osdmap e708: 8 total, 8 up, 8 in 2024-09-16T09:00:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:57 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:57 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:57 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T09:00:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:57 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144"}]': finished 2024-09-16T09:00:57.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:57 smithi169 ceph-mon[33465]: osdmap e709: 8 total, 8 up, 8 in 2024-09-16T09:00:58.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:58 smithi160 ceph-mon[30137]: pgmap v1094: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:00:58.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:58.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:58 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:58.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:58.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:58 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:58 smithi160 ceph-mon[40028]: pgmap v1094: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:00:58.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:58.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:58 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:58.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:58.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:58 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:58.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:58 smithi169 ceph-mon[33465]: pgmap v1094: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:00:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:58 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:00:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:58 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-144"} : dispatch 2024-09-16T09:00:59.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:00:59 smithi160 ceph-mon[30137]: osdmap e710: 8 total, 8 up, 8 in 2024-09-16T09:00:59.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:00:59 smithi160 ceph-mon[40028]: osdmap e710: 8 total, 8 up, 8 in 2024-09-16T09:00:59.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:00:59 smithi169 ceph-mon[33465]: osdmap e710: 8 total, 8 up, 8 in 2024-09-16T09:01:00.600 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:00 smithi160 ceph-mon[40028]: pgmap v1097: 172 pgs: 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:01:00.601 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:00 smithi160 ceph-mon[40028]: osdmap e711: 8 total, 8 up, 8 in 2024-09-16T09:01:00.601 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:00 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:00.601 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:00 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:00.601 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:00 smithi160 ceph-mon[30137]: pgmap v1097: 172 pgs: 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:01:00.601 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:00 smithi160 ceph-mon[30137]: osdmap e711: 8 total, 8 up, 8 in 2024-09-16T09:01:00.601 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:00 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:00.601 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:00 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:00.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:00 smithi169 ceph-mon[33465]: pgmap v1097: 172 pgs: 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:01:00.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:00 smithi169 ceph-mon[33465]: osdmap e711: 8 total, 8 up, 8 in 2024-09-16T09:01:00.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:00 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:00.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:00 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:01 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:01:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:01 smithi160 ceph-mon[30137]: osdmap e712: 8 total, 8 up, 8 in 2024-09-16T09:01:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:01 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:01:01.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:01 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:01:01.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:01 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:01:01.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:01 smithi160 ceph-mon[40028]: osdmap e712: 8 total, 8 up, 8 in 2024-09-16T09:01:01.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:01 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:01:01.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:01 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:01:01.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:01 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:01:01.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:01 smithi169 ceph-mon[33465]: osdmap e712: 8 total, 8 up, 8 in 2024-09-16T09:01:01.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:01 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:01:01.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:01 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:01:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:02 smithi160 ceph-mon[30137]: pgmap v1100: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:01:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:02 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T09:01:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:02 smithi160 ceph-mon[30137]: osdmap e713: 8 total, 8 up, 8 in 2024-09-16T09:01:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:02 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:02.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:02 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:02.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:02 smithi160 ceph-mon[40028]: pgmap v1100: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:01:02.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:02 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T09:01:02.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:02 smithi160 ceph-mon[40028]: osdmap e713: 8 total, 8 up, 8 in 2024-09-16T09:01:02.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:02 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:02.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:02 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:02.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:02 smithi169 ceph-mon[33465]: pgmap v1100: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:01:02.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:02 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T09:01:02.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:02 smithi169 ceph-mon[33465]: osdmap e713: 8 total, 8 up, 8 in 2024-09-16T09:01:02.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:02 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:02.751 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:02 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:03 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-146"}]': finished 2024-09-16T09:01:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:03 smithi160 ceph-mon[30137]: osdmap e714: 8 total, 8 up, 8 in 2024-09-16T09:01:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:03 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-146", "mode": "writeback"} : dispatch 2024-09-16T09:01:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:03 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-146", "mode": "writeback"} : dispatch 2024-09-16T09:01:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:03 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T09:01:03.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:03 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-146", "mode": "writeback"}]': finished 2024-09-16T09:01:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:03 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-146"}]': finished 2024-09-16T09:01:03.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:03 smithi160 ceph-mon[40028]: osdmap e714: 8 total, 8 up, 8 in 2024-09-16T09:01:03.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:03 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-146", "mode": "writeback"} : dispatch 2024-09-16T09:01:03.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:03 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-146", "mode": "writeback"} : dispatch 2024-09-16T09:01:03.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:03 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T09:01:03.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:03 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-146", "mode": "writeback"}]': finished 2024-09-16T09:01:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:03 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-76079-111", "overlaypool": "test-rados-api-smithi160-76079-146"}]': finished 2024-09-16T09:01:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:03 smithi169 ceph-mon[33465]: osdmap e714: 8 total, 8 up, 8 in 2024-09-16T09:01:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:03 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-146", "mode": "writeback"} : dispatch 2024-09-16T09:01:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:03 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-146", "mode": "writeback"} : dispatch 2024-09-16T09:01:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:03 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-16T09:01:03.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:03 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-76079-146", "mode": "writeback"}]': finished 2024-09-16T09:01:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:04 smithi169 ceph-mon[33465]: pgmap v1103: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:01:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:04 smithi169 ceph-mon[33465]: osdmap e715: 8 total, 8 up, 8 in 2024-09-16T09:01:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:04 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T09:01:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:04 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T09:01:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:04.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:04 smithi160 ceph-mon[30137]: pgmap v1103: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:01:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:04 smithi160 ceph-mon[30137]: osdmap e715: 8 total, 8 up, 8 in 2024-09-16T09:01:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:04 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T09:01:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:04 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T09:01:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:04.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:04 smithi160 ceph-mon[40028]: pgmap v1103: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:01:04.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:04 smithi160 ceph-mon[40028]: osdmap e715: 8 total, 8 up, 8 in 2024-09-16T09:01:04.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:04 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T09:01:04.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:04 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-16T09:01:04.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:05.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:01:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:01:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:01:05.850 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:01:05 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:01:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:05 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_count","val": "2"}]': finished 2024-09-16T09:01:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:05 smithi169 ceph-mon[33465]: osdmap e716: 8 total, 8 up, 8 in 2024-09-16T09:01:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:05 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:01:05.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:05 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:01:05.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:05 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_count","val": "2"}]': finished 2024-09-16T09:01:05.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:05 smithi160 ceph-mon[30137]: osdmap e716: 8 total, 8 up, 8 in 2024-09-16T09:01:05.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:05 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:01:05.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:05 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:01:05.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:05 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_count","val": "2"}]': finished 2024-09-16T09:01:05.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:05 smithi160 ceph-mon[40028]: osdmap e716: 8 total, 8 up, 8 in 2024-09-16T09:01:05.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:05 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:01:05.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:05 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-16T09:01:06.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:06 smithi169 ceph-mon[33465]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-16T09:01:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:06 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_period","val": "600"}]': finished 2024-09-16T09:01:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:06 smithi169 ceph-mon[33465]: osdmap e717: 8 total, 8 up, 8 in 2024-09-16T09:01:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:06 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:01:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:06 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:01:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:06 smithi160 ceph-mon[30137]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-16T09:01:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:06 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_period","val": "600"}]': finished 2024-09-16T09:01:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:06 smithi160 ceph-mon[30137]: osdmap e717: 8 total, 8 up, 8 in 2024-09-16T09:01:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:06 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:01:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:06 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:01:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:06.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:06 smithi160 ceph-mon[40028]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-16T09:01:06.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:06 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_period","val": "600"}]': finished 2024-09-16T09:01:06.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:06 smithi160 ceph-mon[40028]: osdmap e717: 8 total, 8 up, 8 in 2024-09-16T09:01:06.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:06 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:01:06.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:06 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-16T09:01:06.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:07.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:07 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T09:01:07.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:07 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T09:01:07.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:07 smithi169 ceph-mon[33465]: osdmap e718: 8 total, 8 up, 8 in 2024-09-16T09:01:07.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:07 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T09:01:07.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:07 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T09:01:07.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:07 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T09:01:07.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:07 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T09:01:07.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:07 smithi160 ceph-mon[40028]: osdmap e718: 8 total, 8 up, 8 in 2024-09-16T09:01:07.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:07 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T09:01:07.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:07 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T09:01:07.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:07 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-16T09:01:07.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:07 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-16T09:01:07.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:07 smithi160 ceph-mon[30137]: osdmap e718: 8 total, 8 up, 8 in 2024-09-16T09:01:07.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:07 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T09:01:07.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:07 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-16T09:01:08.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:08 smithi169 ceph-mon[33465]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-16T09:01:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:08 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-16T09:01:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:08 smithi169 ceph-mon[33465]: osdmap e719: 8 total, 8 up, 8 in 2024-09-16T09:01:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:08 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-16T09:01:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:08 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-16T09:01:08.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:08 smithi160 ceph-mon[30137]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-16T09:01:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:08 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-16T09:01:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:08 smithi160 ceph-mon[30137]: osdmap e719: 8 total, 8 up, 8 in 2024-09-16T09:01:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:08 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-16T09:01:08.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:08 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-16T09:01:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:08 smithi160 ceph-mon[40028]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-16T09:01:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:08 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-16T09:01:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:08 smithi160 ceph-mon[40028]: osdmap e719: 8 total, 8 up, 8 in 2024-09-16T09:01:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:08 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-16T09:01:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:08 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-16T09:01:09.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:09 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "target_max_objects","val": "1"}]': finished 2024-09-16T09:01:09.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:09 smithi169 ceph-mon[33465]: osdmap e720: 8 total, 8 up, 8 in 2024-09-16T09:01:09.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:09 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "target_max_objects","val": "1"}]': finished 2024-09-16T09:01:09.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:09 smithi160 ceph-mon[30137]: osdmap e720: 8 total, 8 up, 8 in 2024-09-16T09:01:09.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:09 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-146","var": "target_max_objects","val": "1"}]': finished 2024-09-16T09:01:09.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:09 smithi160 ceph-mon[40028]: osdmap e720: 8 total, 8 up, 8 in 2024-09-16T09:01:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:10 smithi169 ceph-mon[33465]: pgmap v1112: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:01:10.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:10 smithi169 ceph-mon[33465]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-16T09:01:10.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:10 smithi160 ceph-mon[30137]: pgmap v1112: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:01:10.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:10 smithi160 ceph-mon[30137]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-16T09:01:10.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:10 smithi160 ceph-mon[40028]: pgmap v1112: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:01:10.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:10 smithi160 ceph-mon[40028]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-16T09:01:12.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:12 smithi169 ceph-mon[33465]: pgmap v1113: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 872 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:01:12.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:12 smithi160 ceph-mon[30137]: pgmap v1113: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 872 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:01:12.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:12 smithi160 ceph-mon[40028]: pgmap v1113: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 872 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:01:14.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:14 smithi169 ceph-mon[33465]: pgmap v1114: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 745 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:01:14.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:14 smithi160 ceph-mon[30137]: pgmap v1114: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 745 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:01:14.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:14 smithi160 ceph-mon[40028]: pgmap v1114: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 745 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:01:15.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:01:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:01:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:01:15.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:01:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:01:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:16 smithi169 ceph-mon[33465]: pgmap v1115: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:01:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:16.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:16 smithi160 ceph-mon[40028]: pgmap v1115: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:01:16.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:16 smithi160 ceph-mon[30137]: pgmap v1115: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:01:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:18.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:18 smithi169 ceph-mon[33465]: pgmap v1116: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:01:18.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:18 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:18.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:18 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:18.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:18 smithi160 ceph-mon[30137]: pgmap v1116: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:01:18.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:18 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:18.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:18 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:18.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:18 smithi160 ceph-mon[40028]: pgmap v1116: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T09:01:18.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:18 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:18.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:18 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:19.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:19 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T09:01:19.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:19 smithi169 ceph-mon[33465]: osdmap e721: 8 total, 8 up, 8 in 2024-09-16T09:01:19.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:19 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:19.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:19 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:19.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:19.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:19.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:19 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T09:01:19.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:19 smithi160 ceph-mon[30137]: osdmap e721: 8 total, 8 up, 8 in 2024-09-16T09:01:19.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:19 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:19.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:19 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:19.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:19.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:19.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:19 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T09:01:19.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:19 smithi160 ceph-mon[40028]: osdmap e721: 8 total, 8 up, 8 in 2024-09-16T09:01:19.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:19 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:19.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:19 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:19.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:19.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:20.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:20 smithi169 ceph-mon[33465]: pgmap v1118: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:01:20.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:20 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146"}]': finished 2024-09-16T09:01:20.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:20 smithi169 ceph-mon[33465]: osdmap e722: 8 total, 8 up, 8 in 2024-09-16T09:01:20.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:20.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:20 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:20.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:20 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:20.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:20 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:20.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:20 smithi160 ceph-mon[30137]: pgmap v1118: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:01:20.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:20 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146"}]': finished 2024-09-16T09:01:20.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:20 smithi160 ceph-mon[30137]: osdmap e722: 8 total, 8 up, 8 in 2024-09-16T09:01:20.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:20.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:20 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:20.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:20 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:20.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:20 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:20.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:20 smithi160 ceph-mon[40028]: pgmap v1118: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:01:20.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:20 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146"}]': finished 2024-09-16T09:01:20.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:20 smithi160 ceph-mon[40028]: osdmap e722: 8 total, 8 up, 8 in 2024-09-16T09:01:20.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:20.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:20 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:20.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:20 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:20.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:20 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-146"} : dispatch 2024-09-16T09:01:21.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:21 smithi169 ceph-mon[33465]: osdmap e723: 8 total, 8 up, 8 in 2024-09-16T09:01:21.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:21 smithi160 ceph-mon[30137]: osdmap e723: 8 total, 8 up, 8 in 2024-09-16T09:01:21.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:21 smithi160 ceph-mon[40028]: osdmap e723: 8 total, 8 up, 8 in 2024-09-16T09:01:22.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:22 smithi169 ceph-mon[33465]: pgmap v1121: 172 pgs: 172 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-16T09:01:22.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:22 smithi169 ceph-mon[33465]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-16T09:01:22.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:22 smithi169 ceph-mon[33465]: osdmap e724: 8 total, 8 up, 8 in 2024-09-16T09:01:22.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:22 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:22.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:22 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:22.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:22 smithi160 ceph-mon[40028]: pgmap v1121: 172 pgs: 172 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-16T09:01:22.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:22 smithi160 ceph-mon[40028]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-16T09:01:22.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:22 smithi160 ceph-mon[40028]: osdmap e724: 8 total, 8 up, 8 in 2024-09-16T09:01:22.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:22 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:22.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:22 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:22.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:22 smithi160 ceph-mon[30137]: pgmap v1121: 172 pgs: 172 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-16T09:01:22.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:22 smithi160 ceph-mon[30137]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-16T09:01:22.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:22 smithi160 ceph-mon[30137]: osdmap e724: 8 total, 8 up, 8 in 2024-09-16T09:01:22.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:22 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:22.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:22 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:23.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:23 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:01:23.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:23 smithi169 ceph-mon[33465]: osdmap e725: 8 total, 8 up, 8 in 2024-09-16T09:01:23.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:23 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:01:23.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:23 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:01:23.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:23 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:01:23.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:23 smithi160 ceph-mon[30137]: osdmap e725: 8 total, 8 up, 8 in 2024-09-16T09:01:23.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:23 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:01:23.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:23 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:01:23.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:23 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:01:23.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:23 smithi160 ceph-mon[40028]: osdmap e725: 8 total, 8 up, 8 in 2024-09-16T09:01:23.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:23 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:01:23.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:23 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-16T09:01:24.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:24 smithi169 ceph-mon[33465]: pgmap v1124: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:01:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:24 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T09:01:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:24 smithi169 ceph-mon[33465]: osdmap e726: 8 total, 8 up, 8 in 2024-09-16T09:01:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:24 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148"} : dispatch 2024-09-16T09:01:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:24 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148"} : dispatch 2024-09-16T09:01:24.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:24 smithi160 ceph-mon[40028]: pgmap v1124: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:01:24.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:24 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T09:01:24.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:24 smithi160 ceph-mon[40028]: osdmap e726: 8 total, 8 up, 8 in 2024-09-16T09:01:24.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:24 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148"} : dispatch 2024-09-16T09:01:24.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:24 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148"} : dispatch 2024-09-16T09:01:24.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:24 smithi160 ceph-mon[30137]: pgmap v1124: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:01:24.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:24 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-16T09:01:24.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:24 smithi160 ceph-mon[30137]: osdmap e726: 8 total, 8 up, 8 in 2024-09-16T09:01:24.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:24 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148"} : dispatch 2024-09-16T09:01:24.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:24 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148"} : dispatch 2024-09-16T09:01:25.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:01:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:01:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:01:25.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:01:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:01:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:25 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148"}]': finished 2024-09-16T09:01:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:25 smithi169 ceph-mon[33465]: osdmap e727: 8 total, 8 up, 8 in 2024-09-16T09:01:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:25 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:25 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148"} : dispatch 2024-09-16T09:01:25.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:25 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148"} : dispatch 2024-09-16T09:01:25.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:25 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148"}]': finished 2024-09-16T09:01:25.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:25 smithi160 ceph-mon[30137]: osdmap e727: 8 total, 8 up, 8 in 2024-09-16T09:01:25.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:25.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:25 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:25.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:25 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148"} : dispatch 2024-09-16T09:01:25.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:25 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148"} : dispatch 2024-09-16T09:01:25.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:25 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148"}]': finished 2024-09-16T09:01:25.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:25 smithi160 ceph-mon[40028]: osdmap e727: 8 total, 8 up, 8 in 2024-09-16T09:01:25.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:25.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:25 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:25.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:25 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148"} : dispatch 2024-09-16T09:01:25.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:25 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-148"} : dispatch 2024-09-16T09:01:26.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:26 smithi169 ceph-mon[33465]: pgmap v1127: 204 pgs: 204 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T09:01:26.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:26.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:26 smithi169 ceph-mon[33465]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:01:26.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:26 smithi169 ceph-mon[33465]: osdmap e728: 8 total, 8 up, 8 in 2024-09-16T09:01:26.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:26 smithi160 ceph-mon[30137]: pgmap v1127: 204 pgs: 204 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T09:01:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:26 smithi160 ceph-mon[30137]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:01:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:26 smithi160 ceph-mon[30137]: osdmap e728: 8 total, 8 up, 8 in 2024-09-16T09:01:26.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:26 smithi160 ceph-mon[40028]: pgmap v1127: 204 pgs: 204 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T09:01:26.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:26.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:26 smithi160 ceph-mon[40028]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:01:26.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:26 smithi160 ceph-mon[40028]: osdmap e728: 8 total, 8 up, 8 in 2024-09-16T09:01:27.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:27 smithi169 ceph-mon[33465]: osdmap e729: 8 total, 8 up, 8 in 2024-09-16T09:01:27.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:27 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:27.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:27 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:27.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:27 smithi160 ceph-mon[40028]: osdmap e729: 8 total, 8 up, 8 in 2024-09-16T09:01:27.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:27 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:27.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:27 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:27.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:27 smithi160 ceph-mon[30137]: osdmap e729: 8 total, 8 up, 8 in 2024-09-16T09:01:27.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:27 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:27.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:27 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:28 smithi169 ceph-mon[33465]: pgmap v1130: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T09:01:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:28 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:01:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:28 smithi169 ceph-mon[33465]: osdmap e730: 8 total, 8 up, 8 in 2024-09-16T09:01:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:28 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:28 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-150"} : dispatch 2024-09-16T09:01:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:28 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-150"} : dispatch 2024-09-16T09:01:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:28 smithi160 ceph-mon[30137]: pgmap v1130: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T09:01:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:28 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:01:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:28 smithi160 ceph-mon[30137]: osdmap e730: 8 total, 8 up, 8 in 2024-09-16T09:01:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:28 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:28 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-150"} : dispatch 2024-09-16T09:01:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:28 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-150"} : dispatch 2024-09-16T09:01:28.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:28 smithi160 ceph-mon[40028]: pgmap v1130: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T09:01:28.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:28 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:01:28.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:28 smithi160 ceph-mon[40028]: osdmap e730: 8 total, 8 up, 8 in 2024-09-16T09:01:28.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:28.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:28 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:28.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:28 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-150"} : dispatch 2024-09-16T09:01:28.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:28 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-150"} : dispatch 2024-09-16T09:01:29.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:29 smithi160 ceph-mon[30137]: osdmap e731: 8 total, 8 up, 8 in 2024-09-16T09:01:29.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:29 smithi160 ceph-mon[30137]: osdmap e732: 8 total, 8 up, 8 in 2024-09-16T09:01:29.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:29 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:29.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:29 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:29.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:29 smithi160 ceph-mon[40028]: osdmap e731: 8 total, 8 up, 8 in 2024-09-16T09:01:29.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:29 smithi160 ceph-mon[40028]: osdmap e732: 8 total, 8 up, 8 in 2024-09-16T09:01:29.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:29 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:29.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:29 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:30.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:29 smithi169 ceph-mon[33465]: osdmap e731: 8 total, 8 up, 8 in 2024-09-16T09:01:30.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:29 smithi169 ceph-mon[33465]: osdmap e732: 8 total, 8 up, 8 in 2024-09-16T09:01:30.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:29 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:30.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:29 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:30.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:30 smithi160 ceph-mon[30137]: pgmap v1133: 172 pgs: 172 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-16T09:01:30.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:30 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:01:30.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:30 smithi160 ceph-mon[30137]: osdmap e733: 8 total, 8 up, 8 in 2024-09-16T09:01:30.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:30 smithi160 ceph-mon[40028]: pgmap v1133: 172 pgs: 172 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-16T09:01:30.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:30 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:01:30.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:30 smithi160 ceph-mon[40028]: osdmap e733: 8 total, 8 up, 8 in 2024-09-16T09:01:31.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:30 smithi169 ceph-mon[33465]: pgmap v1133: 172 pgs: 172 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-16T09:01:31.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:30 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:01:31.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:30 smithi169 ceph-mon[33465]: osdmap e733: 8 total, 8 up, 8 in 2024-09-16T09:01:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-152"} : dispatch 2024-09-16T09:01:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-152"} : dispatch 2024-09-16T09:01:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:01:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:31.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:31.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[30137]: osdmap e734: 8 total, 8 up, 8 in 2024-09-16T09:01:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-152"} : dispatch 2024-09-16T09:01:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-152"} : dispatch 2024-09-16T09:01:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:01:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:31.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:31 smithi160 ceph-mon[40028]: osdmap e734: 8 total, 8 up, 8 in 2024-09-16T09:01:32.001 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:32.001 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:31 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:32.001 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-152"} : dispatch 2024-09-16T09:01:32.001 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:31 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-152"} : dispatch 2024-09-16T09:01:32.001 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:31 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:01:32.002 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:31 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:32.002 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:31 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:32.002 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:31 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:32.002 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:31 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:32.002 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:31 smithi169 ceph-mon[33465]: osdmap e734: 8 total, 8 up, 8 in 2024-09-16T09:01:32.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:32 smithi160 ceph-mon[40028]: pgmap v1136: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-16T09:01:32.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:32 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:01:32.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:32 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:01:32.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:32 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:32.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:32 smithi160 ceph-mon[40028]: osdmap e735: 8 total, 8 up, 8 in 2024-09-16T09:01:32.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:32.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:32 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:32.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:32 smithi160 ceph-mon[30137]: pgmap v1136: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-16T09:01:32.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:32 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:01:32.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:32 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:01:32.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:32 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:32.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:32 smithi160 ceph-mon[30137]: osdmap e735: 8 total, 8 up, 8 in 2024-09-16T09:01:32.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:32.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:32 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:33.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:32 smithi169 ceph-mon[33465]: pgmap v1136: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-16T09:01:33.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:32 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:01:33.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:32 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:01:33.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:32 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:33.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:32 smithi169 ceph-mon[33465]: osdmap e735: 8 total, 8 up, 8 in 2024-09-16T09:01:33.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:33.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:32 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-16T09:01:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:34 smithi169 ceph-mon[33465]: pgmap v1139: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail 2024-09-16T09:01:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:34 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:01:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:34 smithi169 ceph-mon[33465]: osdmap e736: 8 total, 8 up, 8 in 2024-09-16T09:01:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-111","var": "dedup_tier","val": "test-rados-api-smithi160-76079-154"} : dispatch 2024-09-16T09:01:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:34 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-111","var": "dedup_tier","val": "test-rados-api-smithi160-76079-154"} : dispatch 2024-09-16T09:01:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:34 smithi169 ceph-mon[33465]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:01:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:34 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:34.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:34 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-154"} : dispatch 2024-09-16T09:01:34.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:34 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-154"} : dispatch 2024-09-16T09:01:34.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:34.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:34.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[30137]: pgmap v1139: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail 2024-09-16T09:01:34.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:01:34.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[30137]: osdmap e736: 8 total, 8 up, 8 in 2024-09-16T09:01:34.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-111","var": "dedup_tier","val": "test-rados-api-smithi160-76079-154"} : dispatch 2024-09-16T09:01:34.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-111","var": "dedup_tier","val": "test-rados-api-smithi160-76079-154"} : dispatch 2024-09-16T09:01:34.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[30137]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:01:34.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:34.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:34.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-154"} : dispatch 2024-09-16T09:01:34.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-154"} : dispatch 2024-09-16T09:01:34.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:34.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:34.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[40028]: pgmap v1139: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail 2024-09-16T09:01:34.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-76079-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-16T09:01:34.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[40028]: osdmap e736: 8 total, 8 up, 8 in 2024-09-16T09:01:34.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-111","var": "dedup_tier","val": "test-rados-api-smithi160-76079-154"} : dispatch 2024-09-16T09:01:34.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi160-76079-111","var": "dedup_tier","val": "test-rados-api-smithi160-76079-154"} : dispatch 2024-09-16T09:01:34.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[40028]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:01:34.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:34.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:34.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-154"} : dispatch 2024-09-16T09:01:34.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi160-76079-111", "tierpool": "test-rados-api-smithi160-76079-154"} : dispatch 2024-09-16T09:01:34.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:01:34.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:35.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:01:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:01:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:01:35.850 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:01:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:01:35.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:35 smithi169 ceph-mon[33465]: osdmap e737: 8 total, 8 up, 8 in 2024-09-16T09:01:35.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:35 smithi160 ceph-mon[40028]: osdmap e737: 8 total, 8 up, 8 in 2024-09-16T09:01:35.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:35 smithi160 ceph-mon[30137]: osdmap e737: 8 total, 8 up, 8 in 2024-09-16T09:01:36.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:36 smithi160 ceph-mon[40028]: pgmap v1142: 172 pgs: 172 active+clean; 4.4 MiB data, 799 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-16T09:01:36.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:36 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:36.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:36 smithi160 ceph-mon[40028]: osdmap e738: 8 total, 8 up, 8 in 2024-09-16T09:01:36.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:36 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:36.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:36 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:36.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:36 smithi160 ceph-mon[30137]: pgmap v1142: 172 pgs: 172 active+clean; 4.4 MiB data, 799 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-16T09:01:36.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:36 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:36.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:36 smithi160 ceph-mon[30137]: osdmap e738: 8 total, 8 up, 8 in 2024-09-16T09:01:36.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:36 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:36.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:36 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:37.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:36 smithi169 ceph-mon[33465]: pgmap v1142: 172 pgs: 172 active+clean; 4.4 MiB data, 799 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-16T09:01:37.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:36 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:37.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:36 smithi169 ceph-mon[33465]: osdmap e738: 8 total, 8 up, 8 in 2024-09-16T09:01:37.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:36 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:37.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:36 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:37.636 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7838 ms) 2024-09-16T09:01:37.637 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-09-16T09:01:37.637 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (13061 ms) 2024-09-16T09:01:37.637 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-09-16T09:01:37.637 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8057 ms) 2024-09-16T09:01:37.637 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-09-16T09:01:37.637 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13079 ms) 2024-09-16T09:01:37.637 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-09-16T09:01:37.637 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7823 ms) 2024-09-16T09:01:37.637 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-09-16T09:01:37.637 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8048 ms) 2024-09-16T09:01:37.637 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-09-16T09:01:37.637 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-16T09:01:37.637 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: ok, hit_set contains 336:602f83fe:::foo:head 2024-09-16T09:01:37.638 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9074 ms) 2024-09-16T09:01:37.638 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-09-16T09:01:37.638 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726477212,0 2024-09-16T09:01:37.638 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: first is 1726477212 2024-09-16T09:01:37.638 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726477212,0 2024-09-16T09:01:37.638 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726477212,0 2024-09-16T09:01:37.638 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726477212,1726477214,1726477215,0 2024-09-16T09:01:37.638 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726477212,1726477214,1726477215,0 2024-09-16T09:01:37.638 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726477212,1726477214,1726477215,0 2024-09-16T09:01:37.638 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726477212,1726477214,1726477215,1726477217,1726477218,0 2024-09-16T09:01:37.638 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726477212,1726477214,1726477215,1726477217,1726477218,0 2024-09-16T09:01:37.638 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726477212,1726477214,1726477215,1726477217,1726477218,0 2024-09-16T09:01:37.638 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726477212,1726477214,1726477215,1726477217,1726477218,1726477220,1726477221,0 2024-09-16T09:01:37.638 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726477212,1726477214,1726477215,1726477217,1726477218,1726477220,1726477221,0 2024-09-16T09:01:37.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726477212,1726477214,1726477215,1726477217,1726477218,1726477220,1726477221,0 2024-09-16T09:01:37.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726477215,1726477217,1726477218,1726477220,1726477221,1726477223,1726477224,0 2024-09-16T09:01:37.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: first now 1726477215, trimmed 2024-09-16T09:01:37.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (21090 ms) 2024-09-16T09:01:37.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-09-16T09:01:37.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: foo0 2024-09-16T09:01:37.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-16T09:01:37.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (15082 ms) 2024-09-16T09:01:37.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-09-16T09:01:37.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (17816 ms) 2024-09-16T09:01:37.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-09-16T09:01:37.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22125 ms) 2024-09-16T09:01:37.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-09-16T09:01:37.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5039 ms) 2024-09-16T09:01:37.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-09-16T09:01:37.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3020 ms) 2024-09-16T09:01:37.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-09-16T09:01:37.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3010 ms) 2024-09-16T09:01:37.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-09-16T09:01:37.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3012 ms) 2024-09-16T09:01:37.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (232820 ms total) 2024-09-16T09:01:37.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: 2024-09-16T09:01:37.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-09-16T09:01:37.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (848978 ms total) 2024-09-16T09:01:37.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-09-16T09:01:37.646 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.646 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75712 2024-09-16T09:01:37.646 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 75712 2024-09-16T09:01:37.646 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.646 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75915 2024-09-16T09:01:37.646 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 75915 2024-09-16T09:01:37.646 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.646 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76490 2024-09-16T09:01:37.646 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76490 2024-09-16T09:01:37.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76164 2024-09-16T09:01:37.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76164 2024-09-16T09:01:37.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76044 2024-09-16T09:01:37.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76044 2024-09-16T09:01:37.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75790 2024-09-16T09:01:37.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 75790 2024-09-16T09:01:37.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76116 2024-09-16T09:01:37.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76116 2024-09-16T09:01:37.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76590 2024-09-16T09:01:37.648 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76590 2024-09-16T09:01:37.648 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.648 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76643 2024-09-16T09:01:37.648 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76643 2024-09-16T09:01:37.648 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.648 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75812 2024-09-16T09:01:37.648 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 75812 2024-09-16T09:01:37.648 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.648 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76512 2024-09-16T09:01:37.648 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76512 2024-09-16T09:01:37.648 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.648 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75687 2024-09-16T09:01:37.648 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 75687 2024-09-16T09:01:37.648 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.649 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75749 2024-09-16T09:01:37.649 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 75749 2024-09-16T09:01:37.649 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.649 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76426 2024-09-16T09:01:37.649 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76426 2024-09-16T09:01:37.649 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.649 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75678 2024-09-16T09:01:37.649 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 75678 2024-09-16T09:01:37.649 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.649 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75698 2024-09-16T09:01:37.649 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 75698 2024-09-16T09:01:37.649 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.649 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76806 2024-09-16T09:01:37.650 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76806 2024-09-16T09:01:37.650 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.650 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75952 2024-09-16T09:01:37.650 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 75952 2024-09-16T09:01:37.650 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.650 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76904 2024-09-16T09:01:37.650 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76904 2024-09-16T09:01:37.650 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.650 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76095 2024-09-16T09:01:37.650 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76095 2024-09-16T09:01:37.650 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.650 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76574 2024-09-16T09:01:37.650 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76574 2024-09-16T09:01:37.650 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.651 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76977 2024-09-16T09:01:37.651 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76977 2024-09-16T09:01:37.651 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.651 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76027 2024-09-16T09:01:37.651 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76027 2024-09-16T09:01:37.651 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.651 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76461 2024-09-16T09:01:37.651 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76461 2024-09-16T09:01:37.651 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.651 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76764 2024-09-16T09:01:37.651 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76764 2024-09-16T09:01:37.651 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.651 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76559 2024-09-16T09:01:37.652 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76559 2024-09-16T09:01:37.652 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.652 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76862 2024-09-16T09:01:37.652 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76862 2024-09-16T09:01:37.652 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.652 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76370 2024-09-16T09:01:37.652 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76370 2024-09-16T09:01:37.652 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.652 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75861 2024-09-16T09:01:37.652 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 75861 2024-09-16T09:01:37.652 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.652 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76926 2024-09-16T09:01:37.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76926 2024-09-16T09:01:37.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76245 2024-09-16T09:01:37.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76245 2024-09-16T09:01:37.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75671 2024-09-16T09:01:37.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 75671 2024-09-16T09:01:37.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75730 2024-09-16T09:01:37.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 75730 2024-09-16T09:01:37.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76147 2024-09-16T09:01:37.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76147 2024-09-16T09:01:37.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=75772 2024-09-16T09:01:37.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 75772 2024-09-16T09:01:37.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76270 2024-09-16T09:01:37.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76270 2024-09-16T09:01:37.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76303 2024-09-16T09:01:37.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76303 2024-09-16T09:01:37.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=76699 2024-09-16T09:01:37.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 76699 2024-09-16T09:01:37.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-16T09:01:37.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=77048 2024-09-16T09:01:37.655 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 77048 2024-09-16T09:01:37.655 INFO:tasks.workunit.client.0.smithi160.stderr:+ exit 0 2024-09-16T09:01:37.655 INFO:tasks.workunit.client.0.smithi160.stderr:+ cleanup 2024-09-16T09:01:37.655 INFO:tasks.workunit.client.0.smithi160.stderr:+ pkill -P 75665 2024-09-16T09:01:37.667 INFO:tasks.workunit.client.0.smithi160.stderr:+ true 2024-09-16T09:01:37.668 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T09:01:37.668 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T09:01:37.701 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-09-16T09:01:37.702 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-09-16T09:01:37.703 DEBUG:teuthology.orchestra.run.smithi160:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8293d73f8690540e843a81caec373f9cc29cf705 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-09-16T09:01:37.762 INFO:tasks.workunit.client.0.smithi160.stderr:++ uuidgen 2024-09-16T09:01:37.768 INFO:tasks.workunit.client.0.smithi160.stderr:+ p=0356ba9f-a8cb-465e-96c7-f3ebcc55b3da 2024-09-16T09:01:37.768 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph osd pool create 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da 12 2024-09-16T09:01:37.854 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.853+0000 7fd116e5a640 1 -- 172.21.15.160:0/194950 >> v1:172.21.15.160:6789/0 conn(0x7fd11010f2a0 legacy=0x7fd110111740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:37.854 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.853+0000 7fd116e5a640 1 -- 172.21.15.160:0/194950 shutdown_connections 2024-09-16T09:01:37.854 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.853+0000 7fd116e5a640 1 -- 172.21.15.160:0/194950 >> 172.21.15.160:0/194950 conn(0x7fd1100fe5b0 msgr2=0x7fd1101009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:37.854 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.853+0000 7fd116e5a640 1 -- 172.21.15.160:0/194950 shutdown_connections 2024-09-16T09:01:37.854 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.853+0000 7fd116e5a640 1 -- 172.21.15.160:0/194950 wait complete. 2024-09-16T09:01:37.854 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.853+0000 7fd116e5a640 1 Processor -- start 2024-09-16T09:01:37.854 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.853+0000 7fd116e5a640 1 -- start start 2024-09-16T09:01:37.854 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd116e5a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd1101a77b0 con 0x7fd11010b6d0 2024-09-16T09:01:37.854 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd116e5a640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd1101a89b0 con 0x7fd110102b70 2024-09-16T09:01:37.855 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd116e5a640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd1101a9bb0 con 0x7fd11010f2a0 2024-09-16T09:01:37.855 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd1153f7640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7fd11010f2a0 0x7fd1101a5eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:36980/0 (socket says 172.21.15.160:36980) 2024-09-16T09:01:37.855 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10ffff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd11010b6d0 0x7fd1101a26d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:38714/0 (socket says 172.21.15.160:38714) 2024-09-16T09:01:37.855 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd1153f7640 1 -- 172.21.15.160:0/3177277888 learned_addr learned my addr 172.21.15.160:0/3177277888 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T09:01:37.857 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4142204226 0 0) 0x7fd1101a77b0 con 0x7fd11010b6d0 2024-09-16T09:01:37.857 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd100003160 con 0x7fd11010b6d0 2024-09-16T09:01:37.857 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 455324915 0 0) 0x7fd1101a9bb0 con 0x7fd11010f2a0 2024-09-16T09:01:37.857 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1101a77b0 con 0x7fd11010f2a0 2024-09-16T09:01:37.857 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1729956425 0 0) 0x7fd1101a89b0 con 0x7fd110102b70 2024-09-16T09:01:37.858 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1101a9bb0 con 0x7fd110102b70 2024-09-16T09:01:37.858 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2601267751 0 0) 0x7fd100003160 con 0x7fd11010b6d0 2024-09-16T09:01:37.858 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd1101a89b0 con 0x7fd11010b6d0 2024-09-16T09:01:37.858 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fd0fc0019e0 con 0x7fd11010b6d0 2024-09-16T09:01:37.858 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2051347070 0 0) 0x7fd1101a77b0 con 0x7fd11010f2a0 2024-09-16T09:01:37.858 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd100003160 con 0x7fd11010f2a0 2024-09-16T09:01:37.858 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fd108001cd0 con 0x7fd11010f2a0 2024-09-16T09:01:37.858 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1870529257 0 0) 0x7fd1101a9bb0 con 0x7fd110102b70 2024-09-16T09:01:37.858 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd1101a77b0 con 0x7fd110102b70 2024-09-16T09:01:37.858 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fd104001830 con 0x7fd110102b70 2024-09-16T09:01:37.858 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 899325552 0 0) 0x7fd1101a89b0 con 0x7fd11010b6d0 2024-09-16T09:01:37.858 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 >> v1:172.21.15.160:6790/0 conn(0x7fd11010f2a0 legacy=0x7fd1101a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:37.858 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 >> v1:172.21.15.169:6789/0 conn(0x7fd110102b70 legacy=0x7fd11010e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:37.858 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1101aae50 con 0x7fd11010b6d0 2024-09-16T09:01:37.859 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.854+0000 7fd116e5a640 1 -- 172.21.15.160:0/3177277888 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd1101a79e0 con 0x7fd11010b6d0 2024-09-16T09:01:37.859 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.855+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fd0fc003a60 con 0x7fd11010b6d0 2024-09-16T09:01:37.859 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.855+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fd0fc005ae0 con 0x7fd11010b6d0 2024-09-16T09:01:37.859 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.855+0000 7fd116e5a640 1 -- 172.21.15.160:0/3177277888 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd1101a7fd0 con 0x7fd11010b6d0 2024-09-16T09:01:37.859 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.856+0000 7fd116e5a640 1 -- 172.21.15.160:0/3177277888 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd110103e10 con 0x7fd11010b6d0 2024-09-16T09:01:37.859 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.856+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7fd0fc0035f0 con 0x7fd11010b6d0 2024-09-16T09:01:37.859 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.857+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(740..740 src has 1..740) ==== 7426+0+0 (unknown 4112506047 0 0) 0x7fd0fc0997c0 con 0x7fd11010b6d0 2024-09-16T09:01:37.861 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:37.861+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fd0fc005ac0 con 0x7fd11010b6d0 2024-09-16T09:01:37.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:37 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T09:01:37.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:37 smithi160 ceph-mon[40028]: osdmap e739: 8 total, 8 up, 8 in 2024-09-16T09:01:37.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:37 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:37.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:37 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:37.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:37 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T09:01:37.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:37 smithi160 ceph-mon[30137]: osdmap e739: 8 total, 8 up, 8 in 2024-09-16T09:01:37.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:37 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:37.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:37 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:38.041 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.041+0000 7fd116e5a640 1 -- 172.21.15.160:0/3177277888 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pg_num": 12} v 0) -- 0x7fd110115030 con 0x7fd11010b6d0 2024-09-16T09:01:38.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:37 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T09:01:38.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:37 smithi169 ceph-mon[33465]: osdmap e739: 8 total, 8 up, 8 in 2024-09-16T09:01:38.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:37 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/139172872' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:38.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:37 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-111"} : dispatch 2024-09-16T09:01:38.638 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.638+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pg_num": 12}]=0 pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' created v741) ==== 176+0+0 (unknown 2795261990 0 0) 0x7fd0fc064940 con 0x7fd11010b6d0 2024-09-16T09:01:38.748 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.747+0000 7fd116e5a640 1 -- 172.21.15.160:0/3177277888 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pg_num": 12} v 0) -- 0x7fd1101a8310 con 0x7fd11010b6d0 2024-09-16T09:01:38.748 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.748+0000 7fd10dffb640 1 -- 172.21.15.160:0/3177277888 <== mon.0 v1:172.21.15.160:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pg_num": 12}]=0 pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' already exists v741) ==== 183+0+0 (unknown 1289155036 0 0) 0x7fd0fc069880 con 0x7fd11010b6d0 2024-09-16T09:01:38.748 INFO:tasks.workunit.client.0.smithi160.stderr:pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' already exists 2024-09-16T09:01:38.751 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.750+0000 7fd116e5a640 1 -- 172.21.15.160:0/3177277888 >> v1:172.21.15.160:6800/2859204856 conn(0x7fd10007bdc0 legacy=0x7fd10007e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:38.751 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.750+0000 7fd116e5a640 1 -- 172.21.15.160:0/3177277888 >> v1:172.21.15.160:6789/0 conn(0x7fd11010b6d0 legacy=0x7fd1101a26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:38.751 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.750+0000 7fd116e5a640 1 -- 172.21.15.160:0/3177277888 shutdown_connections 2024-09-16T09:01:38.751 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.750+0000 7fd116e5a640 1 -- 172.21.15.160:0/3177277888 >> 172.21.15.160:0/3177277888 conn(0x7fd1100fe5b0 msgr2=0x7fd110111f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:38.751 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.750+0000 7fd116e5a640 1 -- 172.21.15.160:0/3177277888 shutdown_connections 2024-09-16T09:01:38.751 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.750+0000 7fd116e5a640 1 -- 172.21.15.160:0/3177277888 wait complete. 2024-09-16T09:01:38.760 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph osd pool set-quota 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da max_objects 10 2024-09-16T09:01:38.845 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.842+0000 7f1b5953d640 1 -- 172.21.15.160:0/353120757 >> v1:172.21.15.160:6790/0 conn(0x7f1b5410a660 legacy=0x7f1b5410ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:38.845 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.842+0000 7f1b5953d640 1 -- 172.21.15.160:0/353120757 shutdown_connections 2024-09-16T09:01:38.845 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.842+0000 7f1b5953d640 1 -- 172.21.15.160:0/353120757 >> 172.21.15.160:0/353120757 conn(0x7f1b540fd540 msgr2=0x7f1b540ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:38.845 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.842+0000 7f1b5953d640 1 -- 172.21.15.160:0/353120757 shutdown_connections 2024-09-16T09:01:38.845 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.842+0000 7f1b5953d640 1 -- 172.21.15.160:0/353120757 wait complete. 2024-09-16T09:01:38.845 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.842+0000 7f1b5953d640 1 Processor -- start 2024-09-16T09:01:38.846 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.842+0000 7f1b5953d640 1 -- start start 2024-09-16T09:01:38.846 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.842+0000 7f1b5953d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1b541a78d0 con 0x7f1b5410e230 2024-09-16T09:01:38.846 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.842+0000 7f1b5953d640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1b541a8ad0 con 0x7f1b54101b00 2024-09-16T09:01:38.846 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.842+0000 7f1b5953d640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1b541a9cd0 con 0x7f1b5410a660 2024-09-16T09:01:38.846 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b4bfff640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f1b5410a660 0x7f1b541a2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:37008/0 (socket says 172.21.15.160:37008) 2024-09-16T09:01:38.846 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b4bfff640 1 -- 172.21.15.160:0/3411733961 learned_addr learned my addr 172.21.15.160:0/3411733961 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T09:01:38.846 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2307842319 0 0) 0x7f1b541a78d0 con 0x7f1b5410e230 2024-09-16T09:01:38.846 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b40003520 con 0x7f1b5410e230 2024-09-16T09:01:38.846 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3310272997 0 0) 0x7f1b541a9cd0 con 0x7f1b5410a660 2024-09-16T09:01:38.846 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b541a78d0 con 0x7f1b5410a660 2024-09-16T09:01:38.846 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1574742613 0 0) 0x7f1b541a8ad0 con 0x7f1b54101b00 2024-09-16T09:01:38.846 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b541a9cd0 con 0x7f1b54101b00 2024-09-16T09:01:38.846 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4188249833 0 0) 0x7f1b541a78d0 con 0x7f1b5410a660 2024-09-16T09:01:38.847 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1b541a8ad0 con 0x7f1b5410a660 2024-09-16T09:01:38.847 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f1b3c004430 con 0x7f1b5410a660 2024-09-16T09:01:38.847 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2869478006 0 0) 0x7f1b40003520 con 0x7f1b5410e230 2024-09-16T09:01:38.847 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1b541a78d0 con 0x7f1b5410e230 2024-09-16T09:01:38.847 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f1b4c001bf0 con 0x7f1b5410e230 2024-09-16T09:01:38.847 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3646492545 0 0) 0x7f1b541a9cd0 con 0x7f1b54101b00 2024-09-16T09:01:38.847 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1b40003520 con 0x7f1b54101b00 2024-09-16T09:01:38.847 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f1b440013b0 con 0x7f1b54101b00 2024-09-16T09:01:38.847 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2161362410 0 0) 0x7f1b541a8ad0 con 0x7f1b5410a660 2024-09-16T09:01:38.847 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 >> v1:172.21.15.169:6789/0 conn(0x7f1b54101b00 legacy=0x7f1b5410da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:38.847 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 >> v1:172.21.15.160:6789/0 conn(0x7f1b5410e230 legacy=0x7f1b541a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:38.847 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b541aaf70 con 0x7f1b5410a660 2024-09-16T09:01:38.847 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b5953d640 1 -- 172.21.15.160:0/3411733961 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1b541a9ea0 con 0x7f1b5410a660 2024-09-16T09:01:38.847 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b5953d640 1 -- 172.21.15.160:0/3411733961 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f1b541aa460 con 0x7f1b5410a660 2024-09-16T09:01:38.848 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f1b3c001b50 con 0x7f1b5410a660 2024-09-16T09:01:38.848 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.843+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f1b3c004b10 con 0x7f1b5410a660 2024-09-16T09:01:38.848 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.844+0000 7f1b5953d640 1 -- 172.21.15.160:0/3411733961 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b34005080 con 0x7f1b5410a660 2024-09-16T09:01:38.848 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.845+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7f1b3c0016e0 con 0x7f1b5410a660 2024-09-16T09:01:38.848 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.845+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(741..741 src has 1..741) ==== 7801+0+0 (unknown 2139026434 0 0) 0x7f1b3c099720 con 0x7f1b5410a660 2024-09-16T09:01:38.850 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:38.849+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f1b3c0646d0 con 0x7f1b5410a660 2024-09-16T09:01:38.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:38 smithi160 ceph-mon[30137]: pgmap v1145: 164 pgs: 164 active+clean; 583 KiB data, 799 MiB used, 714 GiB / 715 GiB avail 2024-09-16T09:01:38.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:38 smithi160 ceph-mon[30137]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T09:01:38.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:38 smithi160 ceph-mon[30137]: osdmap e740: 8 total, 8 up, 8 in 2024-09-16T09:01:38.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:38 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3177277888' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pg_num": 12} : dispatch 2024-09-16T09:01:38.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:38 smithi160 ceph-mon[40028]: pgmap v1145: 164 pgs: 164 active+clean; 583 KiB data, 799 MiB used, 714 GiB / 715 GiB avail 2024-09-16T09:01:38.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:38 smithi160 ceph-mon[40028]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T09:01:38.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:38 smithi160 ceph-mon[40028]: osdmap e740: 8 total, 8 up, 8 in 2024-09-16T09:01:38.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:38 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3177277888' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pg_num": 12} : dispatch 2024-09-16T09:01:39.028 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:39.028+0000 7f1b5953d640 1 -- 172.21.15.160:0/3411733961 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"} v 0) -- 0x7f1b34005370 con 0x7f1b5410a660 2024-09-16T09:01:39.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:38 smithi169 ceph-mon[33465]: pgmap v1145: 164 pgs: 164 active+clean; 583 KiB data, 799 MiB used, 714 GiB / 715 GiB avail 2024-09-16T09:01:39.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:38 smithi169 ceph-mon[33465]: from='client.57263 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi160-76079-111"}]': finished 2024-09-16T09:01:39.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:38 smithi169 ceph-mon[33465]: osdmap e740: 8 total, 8 up, 8 in 2024-09-16T09:01:39.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:38 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3177277888' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pg_num": 12} : dispatch 2024-09-16T09:01:39.650 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:39.650+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.2 v1:172.21.15.160:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da v742) ==== 223+0+0 (unknown 2928099497 0 0) 0x7f1b3c069610 con 0x7f1b5410a660 2024-09-16T09:01:39.760 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:39.760+0000 7f1b5953d640 1 -- 172.21.15.160:0/3411733961 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"} v 0) -- 0x7f1b34005c40 con 0x7f1b5410a660 2024-09-16T09:01:39.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3177277888' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pg_num": 12}]': finished 2024-09-16T09:01:39.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:39 smithi160 ceph-mon[30137]: osdmap e741: 8 total, 8 up, 8 in 2024-09-16T09:01:39.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3177277888' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pg_num": 12} : dispatch 2024-09-16T09:01:39.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:39 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3411733961' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:01:39.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:39 smithi160 ceph-mon[30137]: from='client.57275 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:01:39.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:39 smithi160 ceph-mon[30137]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:01:39.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3177277888' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pg_num": 12}]': finished 2024-09-16T09:01:39.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:39 smithi160 ceph-mon[40028]: osdmap e741: 8 total, 8 up, 8 in 2024-09-16T09:01:39.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3177277888' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pg_num": 12} : dispatch 2024-09-16T09:01:39.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:39 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3411733961' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:01:39.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:39 smithi160 ceph-mon[40028]: from='client.57275 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:01:39.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:39 smithi160 ceph-mon[40028]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:01:40.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3177277888' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pg_num": 12}]': finished 2024-09-16T09:01:40.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:39 smithi169 ceph-mon[33465]: osdmap e741: 8 total, 8 up, 8 in 2024-09-16T09:01:40.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3177277888' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pg_num": 12} : dispatch 2024-09-16T09:01:40.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:39 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3411733961' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:01:40.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:39 smithi169 ceph-mon[33465]: from='client.57275 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:01:40.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:39 smithi169 ceph-mon[33465]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:01:40.652 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.652+0000 7f1b50ff9640 1 -- 172.21.15.160:0/3411733961 <== mon.2 v1:172.21.15.160:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da v743) ==== 223+0+0 (unknown 3888309721 0 0) 0x7f1b3c003ec0 con 0x7f1b5410a660 2024-09-16T09:01:40.653 INFO:tasks.workunit.client.0.smithi160.stderr:set-quota max_objects = 10 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da 2024-09-16T09:01:40.656 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.655+0000 7f1b5953d640 1 -- 172.21.15.160:0/3411733961 >> v1:172.21.15.160:6800/2859204856 conn(0x7f1b4007bca0 legacy=0x7f1b4007e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:40.656 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.655+0000 7f1b5953d640 1 -- 172.21.15.160:0/3411733961 >> v1:172.21.15.160:6790/0 conn(0x7f1b5410a660 legacy=0x7f1b541a2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:40.656 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.655+0000 7f1b5953d640 1 -- 172.21.15.160:0/3411733961 shutdown_connections 2024-09-16T09:01:40.656 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.655+0000 7f1b5953d640 1 -- 172.21.15.160:0/3411733961 >> 172.21.15.160:0/3411733961 conn(0x7f1b540fd540 msgr2=0x7f1b5406af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:40.656 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.655+0000 7f1b5953d640 1 -- 172.21.15.160:0/3411733961 shutdown_connections 2024-09-16T09:01:40.656 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.655+0000 7f1b5953d640 1 -- 172.21.15.160:0/3411733961 wait complete. 2024-09-16T09:01:40.666 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph osd pool application enable 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da rados 2024-09-16T09:01:40.750 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.749+0000 7fe7284d6640 1 -- 172.21.15.160:0/2848563250 >> v1:172.21.15.160:6790/0 conn(0x7fe72010b8e0 legacy=0x7fe72010dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:40.750 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.749+0000 7fe7284d6640 1 -- 172.21.15.160:0/2848563250 shutdown_connections 2024-09-16T09:01:40.750 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.749+0000 7fe7284d6640 1 -- 172.21.15.160:0/2848563250 >> 172.21.15.160:0/2848563250 conn(0x7fe7200fd470 msgr2=0x7fe7200ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:40.750 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.749+0000 7fe7284d6640 1 -- 172.21.15.160:0/2848563250 shutdown_connections 2024-09-16T09:01:40.750 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.749+0000 7fe7284d6640 1 -- 172.21.15.160:0/2848563250 wait complete. 2024-09-16T09:01:40.750 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.749+0000 7fe7284d6640 1 Processor -- start 2024-09-16T09:01:40.750 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.749+0000 7fe7284d6640 1 -- start start 2024-09-16T09:01:40.753 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7284d6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe7201a7880 con 0x7fe72010f3f0 2024-09-16T09:01:40.753 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7284d6640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe7201a8a80 con 0x7fe72010b8e0 2024-09-16T09:01:40.753 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7284d6640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe7201a9c80 con 0x7fe720103900 2024-09-16T09:01:40.753 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe726272640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7fe720103900 0x7fe72010b030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:37018/0 (socket says 172.21.15.160:37018) 2024-09-16T09:01:40.753 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe726272640 1 -- 172.21.15.160:0/2190312480 learned_addr learned my addr 172.21.15.160:0/2190312480 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T09:01:40.753 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2271100166 0 0) 0x7fe7201a9c80 con 0x7fe720103900 2024-09-16T09:01:40.753 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe70c003520 con 0x7fe720103900 2024-09-16T09:01:40.753 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3576161456 0 0) 0x7fe7201a7880 con 0x7fe72010f3f0 2024-09-16T09:01:40.753 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7201a9c80 con 0x7fe72010f3f0 2024-09-16T09:01:40.753 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1320464881 0 0) 0x7fe7201a8a80 con 0x7fe72010b8e0 2024-09-16T09:01:40.753 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7201a7880 con 0x7fe72010b8e0 2024-09-16T09:01:40.754 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4099999444 0 0) 0x7fe70c003520 con 0x7fe720103900 2024-09-16T09:01:40.754 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe7201a8a80 con 0x7fe720103900 2024-09-16T09:01:40.754 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fe71c001a10 con 0x7fe720103900 2024-09-16T09:01:40.754 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 371740738 0 0) 0x7fe7201a9c80 con 0x7fe72010f3f0 2024-09-16T09:01:40.754 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe70c003520 con 0x7fe72010f3f0 2024-09-16T09:01:40.754 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fe718001590 con 0x7fe72010f3f0 2024-09-16T09:01:40.754 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 493782265 0 0) 0x7fe7201a7880 con 0x7fe72010b8e0 2024-09-16T09:01:40.754 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe7201a9c80 con 0x7fe72010b8e0 2024-09-16T09:01:40.754 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fe710001500 con 0x7fe72010b8e0 2024-09-16T09:01:40.754 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1240673174 0 0) 0x7fe7201a8a80 con 0x7fe720103900 2024-09-16T09:01:40.755 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 >> v1:172.21.15.169:6789/0 conn(0x7fe72010b8e0 legacy=0x7fe7201a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:40.755 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 >> v1:172.21.15.160:6789/0 conn(0x7fe72010f3f0 legacy=0x7fe7201a5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:40.755 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7201aaf20 con 0x7fe720103900 2024-09-16T09:01:40.755 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7284d6640 1 -- 172.21.15.160:0/2190312480 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe7201a7ab0 con 0x7fe720103900 2024-09-16T09:01:40.755 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.750+0000 7fe7284d6640 1 -- 172.21.15.160:0/2190312480 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fe7201a80f0 con 0x7fe720103900 2024-09-16T09:01:40.755 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.751+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fe71c001cb0 con 0x7fe720103900 2024-09-16T09:01:40.755 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.751+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fe71c005cc0 con 0x7fe720103900 2024-09-16T09:01:40.755 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.751+0000 7fe7284d6640 1 -- 172.21.15.160:0/2190312480 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7040050b0 con 0x7fe720103900 2024-09-16T09:01:40.755 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.752+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7fe71c0038c0 con 0x7fe720103900 2024-09-16T09:01:40.756 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.753+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(743..743 src has 1..743) ==== 7801+0+0 (unknown 1186172263 0 0) 0x7fe71c099850 con 0x7fe720103900 2024-09-16T09:01:40.757 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.757+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fe71c064840 con 0x7fe720103900 2024-09-16T09:01:40.937 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:40.936+0000 7fe7284d6640 1 -- 172.21.15.160:0/2190312480 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"} v 0) -- 0x7fe7040053a0 con 0x7fe720103900 2024-09-16T09:01:40.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:40 smithi160 ceph-mon[30137]: pgmap v1148: 176 pgs: 8 creating+peering, 4 unknown, 164 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail 2024-09-16T09:01:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:40 smithi160 ceph-mon[30137]: from='client.57275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"}]': finished 2024-09-16T09:01:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:40 smithi160 ceph-mon[30137]: osdmap e742: 8 total, 8 up, 8 in 2024-09-16T09:01:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:40 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3411733961' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:01:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:40 smithi160 ceph-mon[30137]: from='client.57275 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:01:40.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:40 smithi160 ceph-mon[40028]: pgmap v1148: 176 pgs: 8 creating+peering, 4 unknown, 164 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail 2024-09-16T09:01:40.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:40 smithi160 ceph-mon[40028]: from='client.57275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"}]': finished 2024-09-16T09:01:40.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:40 smithi160 ceph-mon[40028]: osdmap e742: 8 total, 8 up, 8 in 2024-09-16T09:01:40.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:40 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3411733961' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:01:40.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:40 smithi160 ceph-mon[40028]: from='client.57275 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:01:41.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:40 smithi169 ceph-mon[33465]: pgmap v1148: 176 pgs: 8 creating+peering, 4 unknown, 164 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail 2024-09-16T09:01:41.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:40 smithi169 ceph-mon[33465]: from='client.57275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"}]': finished 2024-09-16T09:01:41.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:40 smithi169 ceph-mon[33465]: osdmap e742: 8 total, 8 up, 8 in 2024-09-16T09:01:41.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:40 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3411733961' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:01:41.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:40 smithi169 ceph-mon[33465]: from='client.57275 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:01:41.663 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:41.663+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.2 v1:172.21.15.160:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"}]=0 enabled application 'rados' on pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' v744) ==== 213+0+0 (unknown 1175890034 0 0) 0x7fe71c069780 con 0x7fe720103900 2024-09-16T09:01:41.774 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:41.774+0000 7fe7284d6640 1 -- 172.21.15.160:0/2190312480 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"} v 0) -- 0x7fe704005c70 con 0x7fe720103900 2024-09-16T09:01:41.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:41 smithi160 ceph-mon[30137]: from='client.57275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"}]': finished 2024-09-16T09:01:41.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:41 smithi160 ceph-mon[30137]: osdmap e743: 8 total, 8 up, 8 in 2024-09-16T09:01:41.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:41 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2190312480' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"} : dispatch 2024-09-16T09:01:41.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:41 smithi160 ceph-mon[30137]: from='client.57281 ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"} : dispatch 2024-09-16T09:01:41.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:41 smithi160 ceph-mon[40028]: from='client.57275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"}]': finished 2024-09-16T09:01:41.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:41 smithi160 ceph-mon[40028]: osdmap e743: 8 total, 8 up, 8 in 2024-09-16T09:01:41.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:41 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2190312480' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"} : dispatch 2024-09-16T09:01:41.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:41 smithi160 ceph-mon[40028]: from='client.57281 ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"} : dispatch 2024-09-16T09:01:42.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:41 smithi169 ceph-mon[33465]: from='client.57275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "10"}]': finished 2024-09-16T09:01:42.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:41 smithi169 ceph-mon[33465]: osdmap e743: 8 total, 8 up, 8 in 2024-09-16T09:01:42.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:41 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2190312480' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"} : dispatch 2024-09-16T09:01:42.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:41 smithi169 ceph-mon[33465]: from='client.57281 ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"} : dispatch 2024-09-16T09:01:42.665 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:42.664+0000 7fe7177fe640 1 -- 172.21.15.160:0/2190312480 <== mon.2 v1:172.21.15.160:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"}]=0 enabled application 'rados' on pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' v745) ==== 213+0+0 (unknown 2084003476 0 0) 0x7fe71c0057c0 con 0x7fe720103900 2024-09-16T09:01:42.665 INFO:tasks.workunit.client.0.smithi160.stderr:enabled application 'rados' on pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' 2024-09-16T09:01:42.669 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:42.668+0000 7fe7284d6640 1 -- 172.21.15.160:0/2190312480 >> v1:172.21.15.160:6800/2859204856 conn(0x7fe70c07bc80 legacy=0x7fe70c07e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:42.669 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:42.668+0000 7fe7284d6640 1 -- 172.21.15.160:0/2190312480 >> v1:172.21.15.160:6790/0 conn(0x7fe720103900 legacy=0x7fe72010b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:42.669 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:42.668+0000 7fe7284d6640 1 -- 172.21.15.160:0/2190312480 shutdown_connections 2024-09-16T09:01:42.669 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:42.668+0000 7fe7284d6640 1 -- 172.21.15.160:0/2190312480 >> 172.21.15.160:0/2190312480 conn(0x7fe7200fd470 msgr2=0x7fe7200ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:42.669 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:42.668+0000 7fe7284d6640 1 -- 172.21.15.160:0/2190312480 shutdown_connections 2024-09-16T09:01:42.669 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:01:42.668+0000 7fe7284d6640 1 -- 172.21.15.160:0/2190312480 wait complete. 2024-09-16T09:01:42.679 INFO:tasks.workunit.client.0.smithi160.stderr:++ seq 1 10 2024-09-16T09:01:42.688 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:01:42.688 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da put obj1 /etc/passwd 2024-09-16T09:01:42.728 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:01:42.728 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da put obj2 /etc/passwd 2024-09-16T09:01:42.765 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:01:42.766 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da put obj3 /etc/passwd 2024-09-16T09:01:42.801 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:01:42.801 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da put obj4 /etc/passwd 2024-09-16T09:01:42.835 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:01:42.835 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da put obj5 /etc/passwd 2024-09-16T09:01:42.849 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 09:01:42 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=sqlstore.transactions t=2024-09-16T09:01:42.469732075Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T09:01:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:42 smithi169 ceph-mon[33465]: pgmap v1151: 176 pgs: 8 creating+peering, 4 unknown, 164 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:01:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:42 smithi169 ceph-mon[33465]: from='client.57281 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"}]': finished 2024-09-16T09:01:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:42 smithi169 ceph-mon[33465]: osdmap e744: 8 total, 8 up, 8 in 2024-09-16T09:01:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:42 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2190312480' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"} : dispatch 2024-09-16T09:01:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:42 smithi169 ceph-mon[33465]: from='client.57281 ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"} : dispatch 2024-09-16T09:01:42.870 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:01:42.870 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da put obj6 /etc/passwd 2024-09-16T09:01:42.904 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:01:42.904 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da put obj7 /etc/passwd 2024-09-16T09:01:42.938 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:01:42.938 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da put obj8 /etc/passwd 2024-09-16T09:01:42.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:42 smithi160 ceph-mon[30137]: pgmap v1151: 176 pgs: 8 creating+peering, 4 unknown, 164 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:01:42.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:42 smithi160 ceph-mon[30137]: from='client.57281 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"}]': finished 2024-09-16T09:01:42.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:42 smithi160 ceph-mon[30137]: osdmap e744: 8 total, 8 up, 8 in 2024-09-16T09:01:42.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:42 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2190312480' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"} : dispatch 2024-09-16T09:01:42.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:42 smithi160 ceph-mon[30137]: from='client.57281 ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"} : dispatch 2024-09-16T09:01:42.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:42 smithi160 ceph-mon[40028]: pgmap v1151: 176 pgs: 8 creating+peering, 4 unknown, 164 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:01:42.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:42 smithi160 ceph-mon[40028]: from='client.57281 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"}]': finished 2024-09-16T09:01:42.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:42 smithi160 ceph-mon[40028]: osdmap e744: 8 total, 8 up, 8 in 2024-09-16T09:01:42.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:42 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2190312480' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"} : dispatch 2024-09-16T09:01:42.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:42 smithi160 ceph-mon[40028]: from='client.57281 ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"} : dispatch 2024-09-16T09:01:42.972 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:01:42.972 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da put obj9 /etc/passwd 2024-09-16T09:01:43.005 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:01:43.005 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da put obj10 /etc/passwd 2024-09-16T09:01:43.039 INFO:tasks.workunit.client.0.smithi160.stderr:+ sleep 30 2024-09-16T09:01:43.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:43 smithi160 ceph-mon[30137]: from='client.57281 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"}]': finished 2024-09-16T09:01:43.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:43 smithi160 ceph-mon[30137]: osdmap e745: 8 total, 8 up, 8 in 2024-09-16T09:01:43.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:43 smithi160 ceph-mon[40028]: from='client.57281 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"}]': finished 2024-09-16T09:01:43.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:43 smithi160 ceph-mon[40028]: osdmap e745: 8 total, 8 up, 8 in 2024-09-16T09:01:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:43 smithi169 ceph-mon[33465]: from='client.57281 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "app": "rados"}]': finished 2024-09-16T09:01:44.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:43 smithi169 ceph-mon[33465]: osdmap e745: 8 total, 8 up, 8 in 2024-09-16T09:01:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:44 smithi160 ceph-mon[30137]: pgmap v1154: 176 pgs: 8 creating+peering, 4 unknown, 164 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail 2024-09-16T09:01:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:44 smithi160 ceph-mon[40028]: pgmap v1154: 176 pgs: 8 creating+peering, 4 unknown, 164 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail 2024-09-16T09:01:45.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:44 smithi169 ceph-mon[33465]: pgmap v1154: 176 pgs: 8 creating+peering, 4 unknown, 164 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail 2024-09-16T09:01:45.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:01:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:01:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:01:45.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:01:45 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:01:46.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:46 smithi160 ceph-mon[30137]: pgmap v1155: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 900 B/s rd, 23 KiB/s wr, 2 op/s 2024-09-16T09:01:46.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:46 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:46.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:46 smithi160 ceph-mon[40028]: pgmap v1155: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 900 B/s rd, 23 KiB/s wr, 2 op/s 2024-09-16T09:01:46.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:46 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:47.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:46 smithi169 ceph-mon[33465]: pgmap v1155: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 900 B/s rd, 23 KiB/s wr, 2 op/s 2024-09-16T09:01:47.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:46 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:48.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:48 smithi160 ceph-mon[30137]: pgmap v1156: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 765 B/s rd, 19 KiB/s wr, 2 op/s 2024-09-16T09:01:48.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:48 smithi160 ceph-mon[40028]: pgmap v1156: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 765 B/s rd, 19 KiB/s wr, 2 op/s 2024-09-16T09:01:49.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:48 smithi169 ceph-mon[33465]: pgmap v1156: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 765 B/s rd, 19 KiB/s wr, 2 op/s 2024-09-16T09:01:49.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:49 smithi160 ceph-mon[30137]: pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' is full (reached quota's max_objects: 10) 2024-09-16T09:01:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:49 smithi160 ceph-mon[30137]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-16T09:01:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:49 smithi160 ceph-mon[30137]: osdmap e746: 8 total, 8 up, 8 in 2024-09-16T09:01:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:49 smithi160 ceph-mon[40028]: pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' is full (reached quota's max_objects: 10) 2024-09-16T09:01:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:49 smithi160 ceph-mon[40028]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-16T09:01:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:49 smithi160 ceph-mon[40028]: osdmap e746: 8 total, 8 up, 8 in 2024-09-16T09:01:50.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:50.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:49 smithi169 ceph-mon[33465]: pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' is full (reached quota's max_objects: 10) 2024-09-16T09:01:50.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:49 smithi169 ceph-mon[33465]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-16T09:01:50.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:49 smithi169 ceph-mon[33465]: osdmap e746: 8 total, 8 up, 8 in 2024-09-16T09:01:50.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:50 smithi160 ceph-mon[30137]: pgmap v1158: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-09-16T09:01:50.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:50 smithi160 ceph-mon[40028]: pgmap v1158: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-09-16T09:01:51.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:50 smithi169 ceph-mon[33465]: pgmap v1158: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-09-16T09:01:52.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:52 smithi160 ceph-mon[30137]: pgmap v1159: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-09-16T09:01:52.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:52 smithi160 ceph-mon[40028]: pgmap v1159: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-09-16T09:01:53.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:52 smithi169 ceph-mon[33465]: pgmap v1159: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-09-16T09:01:54.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:54 smithi160 ceph-mon[30137]: pgmap v1160: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-16T09:01:54.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:54 smithi160 ceph-mon[40028]: pgmap v1160: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-16T09:01:55.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:54 smithi169 ceph-mon[33465]: pgmap v1160: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-16T09:01:55.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:01:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:01:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:01:55.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:01:55 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:01:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:56 smithi160 ceph-mon[30137]: pgmap v1161: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:01:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:56 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:56.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:56 smithi160 ceph-mon[40028]: pgmap v1161: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:01:56.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:56 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:57.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:56 smithi169 ceph-mon[33465]: pgmap v1161: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:01:57.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:56 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:01:59.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:01:58 smithi169 ceph-mon[33465]: pgmap v1162: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:01:59.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:01:58 smithi160 ceph-mon[30137]: pgmap v1162: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:01:59.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:01:58 smithi160 ceph-mon[40028]: pgmap v1162: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:02:01.000 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:00 smithi160 ceph-mon[30137]: pgmap v1163: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-09-16T09:02:01.000 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:00 smithi160 ceph-mon[40028]: pgmap v1163: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-09-16T09:02:01.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:00 smithi169 ceph-mon[33465]: pgmap v1163: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-09-16T09:02:03.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:02 smithi169 ceph-mon[33465]: pgmap v1164: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:03.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:02 smithi160 ceph-mon[30137]: pgmap v1164: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:03.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:02 smithi160 ceph-mon[40028]: pgmap v1164: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:04.989 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:04 smithi160 ceph-mon[30137]: pgmap v1165: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:04.990 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:02:04.990 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:04.990 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:04 smithi160 ceph-mon[40028]: pgmap v1165: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:04.990 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:02:04.990 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:05.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:04 smithi169 ceph-mon[33465]: pgmap v1165: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:05.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:02:05.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:05.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:02:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:02:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:02:05.849 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:02:05 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:02:07.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:06 smithi169 ceph-mon[33465]: pgmap v1166: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:07.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:07.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:06 smithi160 ceph-mon[30137]: pgmap v1166: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:07.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:07.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:06 smithi160 ceph-mon[40028]: pgmap v1166: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:07.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:09.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:08 smithi169 ceph-mon[33465]: pgmap v1167: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:09.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:08 smithi160 ceph-mon[30137]: pgmap v1167: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:09.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:08 smithi160 ceph-mon[40028]: pgmap v1167: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:11.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:10 smithi169 ceph-mon[33465]: pgmap v1168: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:11.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:10 smithi160 ceph-mon[30137]: pgmap v1168: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:11.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:10 smithi160 ceph-mon[40028]: pgmap v1168: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:12.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:11 smithi169 ceph-mon[33465]: pgmap v1169: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:12.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:11 smithi160 ceph-mon[30137]: pgmap v1169: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:12.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:11 smithi160 ceph-mon[40028]: pgmap v1169: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:13.041 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=145921 2024-09-16T09:02:13.041 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph osd pool set-quota 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da max_objects 100 2024-09-16T09:02:13.041 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da put onemore /etc/passwd 2024-09-16T09:02:13.137 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.135+0000 7ff6edfc2640 1 -- 172.21.15.160:0/1297761746 >> v1:172.21.15.160:6789/0 conn(0x7ff6e810f2b0 legacy=0x7ff6e8111750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:13.137 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.135+0000 7ff6edfc2640 1 -- 172.21.15.160:0/1297761746 shutdown_connections 2024-09-16T09:02:13.137 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.135+0000 7ff6edfc2640 1 -- 172.21.15.160:0/1297761746 >> 172.21.15.160:0/1297761746 conn(0x7ff6e80fe5a0 msgr2=0x7ff6e81009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:13.137 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.135+0000 7ff6edfc2640 1 -- 172.21.15.160:0/1297761746 shutdown_connections 2024-09-16T09:02:13.137 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.135+0000 7ff6edfc2640 1 -- 172.21.15.160:0/1297761746 wait complete. 2024-09-16T09:02:13.137 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.135+0000 7ff6edfc2640 1 Processor -- start 2024-09-16T09:02:13.139 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.136+0000 7ff6edfc2640 1 -- start start 2024-09-16T09:02:13.139 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.136+0000 7ff6edfc2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff6e81a7870 con 0x7ff6e8102b80 2024-09-16T09:02:13.140 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.136+0000 7ff6edfc2640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff6e81a8a70 con 0x7ff6e810f2b0 2024-09-16T09:02:13.140 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.136+0000 7ff6edfc2640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff6e81a9c70 con 0x7ff6e810b6e0 2024-09-16T09:02:13.140 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.136+0000 7ff6e77fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7ff6e8102b80 0x7ff6e810ea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:50630/0 (socket says 172.21.15.160:50630) 2024-09-16T09:02:13.140 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.136+0000 7ff6e77fe640 1 -- 172.21.15.160:0/3522397961 learned_addr learned my addr 172.21.15.160:0/3522397961 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T09:02:13.140 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.136+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1529502072 0 0) 0x7ff6e81a7870 con 0x7ff6e8102b80 2024-09-16T09:02:13.140 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.136+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6d0004920 con 0x7ff6e8102b80 2024-09-16T09:02:13.140 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.136+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2891195849 0 0) 0x7ff6e81a9c70 con 0x7ff6e810b6e0 2024-09-16T09:02:13.140 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.136+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6e81a7870 con 0x7ff6e810b6e0 2024-09-16T09:02:13.140 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.137+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2761642909 0 0) 0x7ff6d0004920 con 0x7ff6e8102b80 2024-09-16T09:02:13.140 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.137+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff6e81a9c70 con 0x7ff6e8102b80 2024-09-16T09:02:13.140 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.137+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ff6d8001e20 con 0x7ff6e8102b80 2024-09-16T09:02:13.140 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.137+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 902488560 0 0) 0x7ff6e81a7870 con 0x7ff6e810b6e0 2024-09-16T09:02:13.141 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.137+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff6d0004920 con 0x7ff6e810b6e0 2024-09-16T09:02:13.141 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.137+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3192755767 0 0) 0x7ff6e81a8a70 con 0x7ff6e810f2b0 2024-09-16T09:02:13.141 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.137+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6e81a7870 con 0x7ff6e810f2b0 2024-09-16T09:02:13.141 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.137+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ff6dc001130 con 0x7ff6e810b6e0 2024-09-16T09:02:13.141 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.137+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2343424696 0 0) 0x7ff6e81a9c70 con 0x7ff6e8102b80 2024-09-16T09:02:13.141 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.137+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 >> v1:172.21.15.160:6790/0 conn(0x7ff6e810b6e0 legacy=0x7ff6e81a26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:13.141 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.137+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 >> v1:172.21.15.169:6789/0 conn(0x7ff6e810f2b0 legacy=0x7ff6e81a5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:13.141 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.137+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e81aaf10 con 0x7ff6e8102b80 2024-09-16T09:02:13.141 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.137+0000 7ff6edfc2640 1 -- 172.21.15.160:0/3522397961 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e81a9ea0 con 0x7ff6e8102b80 2024-09-16T09:02:13.141 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.137+0000 7ff6edfc2640 1 -- 172.21.15.160:0/3522397961 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff6e81aa4e0 con 0x7ff6e8102b80 2024-09-16T09:02:13.141 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.137+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7ff6d80040b0 con 0x7ff6e8102b80 2024-09-16T09:02:13.141 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.137+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ff6d8006140 con 0x7ff6e8102b80 2024-09-16T09:02:13.141 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.138+0000 7ff6edfc2640 1 -- 172.21.15.160:0/3522397961 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d40050b0 con 0x7ff6e8102b80 2024-09-16T09:02:13.142 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.138+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7ff6d8003c40 con 0x7ff6e8102b80 2024-09-16T09:02:13.142 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.139+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(746..746 src has 1..746) ==== 7814+0+0 (unknown 2752137020 0 0) 0x7ff6d8099ec0 con 0x7ff6e8102b80 2024-09-16T09:02:13.142 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.139+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=747}) -- 0x7ff6e81a9c70 con 0x7ff6e8102b80 2024-09-16T09:02:13.145 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.144+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7ff6d8064e60 con 0x7ff6e8102b80 2024-09-16T09:02:13.323 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.322+0000 7ff6edfc2640 1 -- 172.21.15.160:0/3522397961 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "100"} v 0) -- 0x7ff6d40053a0 con 0x7ff6e8102b80 2024-09-16T09:02:13.375 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.374+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da v747) ==== 225+0+0 (unknown 3667788607 0 0) 0x7ff6d8069da0 con 0x7ff6e8102b80 2024-09-16T09:02:13.375 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.374+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.0 v1:172.21.15.160:6789/0 11 ==== osd_map(747..747 src has 1..747) ==== 628+0+0 (unknown 1072402630 0 0) 0x7ff6d8097c60 con 0x7ff6e8102b80 2024-09-16T09:02:13.376 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.374+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=748}) -- 0x7ff6e81a8a70 con 0x7ff6e8102b80 2024-09-16T09:02:13.486 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:13.485+0000 7ff6edfc2640 1 -- 172.21.15.160:0/3522397961 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "100"} v 0) -- 0x7ff6d4005c70 con 0x7ff6e8102b80 2024-09-16T09:02:13.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:13 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3522397961' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "100"} : dispatch 2024-09-16T09:02:13.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:13 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3522397961' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "100"} : dispatch 2024-09-16T09:02:13.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:13 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3522397961' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "100"} : dispatch 2024-09-16T09:02:14.309 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.307+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.0 v1:172.21.15.160:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da v748) ==== 225+0+0 (unknown 3651159 0 0) 0x7ff6d8005b40 con 0x7ff6e8102b80 2024-09-16T09:02:14.309 INFO:tasks.workunit.client.0.smithi160.stderr:set-quota max_objects = 100 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da 2024-09-16T09:02:14.310 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.308+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 <== mon.0 v1:172.21.15.160:6789/0 13 ==== osd_map(748..748 src has 1..748) ==== 628+0+0 (unknown 2624295656 0 0) 0x7ff6d805e2d0 con 0x7ff6e8102b80 2024-09-16T09:02:14.310 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.308+0000 7ff6e4ff9640 1 -- 172.21.15.160:0/3522397961 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=749}) -- 0x7ff6d0088e30 con 0x7ff6e8102b80 2024-09-16T09:02:14.313 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.311+0000 7ff6edfc2640 1 -- 172.21.15.160:0/3522397961 >> v1:172.21.15.160:6800/2859204856 conn(0x7ff6d007bc20 legacy=0x7ff6d007e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:14.313 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.312+0000 7ff6edfc2640 1 -- 172.21.15.160:0/3522397961 >> v1:172.21.15.160:6789/0 conn(0x7ff6e8102b80 legacy=0x7ff6e810ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:14.313 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.312+0000 7ff6edfc2640 1 -- 172.21.15.160:0/3522397961 shutdown_connections 2024-09-16T09:02:14.313 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.312+0000 7ff6edfc2640 1 -- 172.21.15.160:0/3522397961 >> 172.21.15.160:0/3522397961 conn(0x7ff6e80fe5a0 msgr2=0x7ff6e8106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:14.314 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.312+0000 7ff6edfc2640 1 -- 172.21.15.160:0/3522397961 shutdown_connections 2024-09-16T09:02:14.314 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.312+0000 7ff6edfc2640 1 -- 172.21.15.160:0/3522397961 wait complete. 2024-09-16T09:02:14.327 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 145921 2024-09-16T09:02:14.327 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 0 -ne 0 ']' 2024-09-16T09:02:14.327 INFO:tasks.workunit.client.0.smithi160.stderr:+ true 2024-09-16T09:02:14.327 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da put twomore /etc/passwd 2024-09-16T09:02:14.364 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph osd pool set-quota 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da max_bytes 100 2024-09-16T09:02:14.453 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.448+0000 7f432ed17640 1 -- 172.21.15.160:0/360911617 >> v1:172.21.15.160:6790/0 conn(0x7f432810a5d0 legacy=0x7f432810c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:14.453 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.448+0000 7f432ed17640 1 -- 172.21.15.160:0/360911617 shutdown_connections 2024-09-16T09:02:14.453 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.448+0000 7f432ed17640 1 -- 172.21.15.160:0/360911617 >> 172.21.15.160:0/360911617 conn(0x7f43280fd470 msgr2=0x7f43280ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:14.453 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.448+0000 7f432ed17640 1 -- 172.21.15.160:0/360911617 shutdown_connections 2024-09-16T09:02:14.453 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.448+0000 7f432ed17640 1 -- 172.21.15.160:0/360911617 wait complete. 2024-09-16T09:02:14.453 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.449+0000 7f432ed17640 1 Processor -- start 2024-09-16T09:02:14.453 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.449+0000 7f432ed17640 1 -- start start 2024-09-16T09:02:14.453 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.449+0000 7f432ed17640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f43281a77e0 con 0x7f4328101a70 2024-09-16T09:02:14.453 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.449+0000 7f432ed17640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f43281a89e0 con 0x7f432810a5d0 2024-09-16T09:02:14.453 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.449+0000 7f432ed17640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f43281a9be0 con 0x7f432810e1a0 2024-09-16T09:02:14.454 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.449+0000 7f432cab3640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f4328101a70 0x7f432810d970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:50664/0 (socket says 172.21.15.160:50664) 2024-09-16T09:02:14.454 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.449+0000 7f432cab3640 1 -- 172.21.15.160:0/2391402354 learned_addr learned my addr 172.21.15.160:0/2391402354 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T09:02:14.454 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 488994225 0 0) 0x7f43281a77e0 con 0x7f4328101a70 2024-09-16T09:02:14.454 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4318003520 con 0x7f4328101a70 2024-09-16T09:02:14.454 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 970409559 0 0) 0x7f43281a9be0 con 0x7f432810e1a0 2024-09-16T09:02:14.454 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43281a77e0 con 0x7f432810e1a0 2024-09-16T09:02:14.454 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 726010251 0 0) 0x7f43281a89e0 con 0x7f432810a5d0 2024-09-16T09:02:14.454 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43281a9be0 con 0x7f432810a5d0 2024-09-16T09:02:14.454 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 979571912 0 0) 0x7f4318003520 con 0x7f4328101a70 2024-09-16T09:02:14.454 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f43281a89e0 con 0x7f4328101a70 2024-09-16T09:02:14.454 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f431c001200 con 0x7f4328101a70 2024-09-16T09:02:14.454 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1655586743 0 0) 0x7f43281a77e0 con 0x7f432810e1a0 2024-09-16T09:02:14.454 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4318003520 con 0x7f432810e1a0 2024-09-16T09:02:14.455 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f4320001bf0 con 0x7f432810e1a0 2024-09-16T09:02:14.455 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1916319096 0 0) 0x7f43281a9be0 con 0x7f432810a5d0 2024-09-16T09:02:14.455 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f43281a77e0 con 0x7f432810a5d0 2024-09-16T09:02:14.455 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f43140014b0 con 0x7f432810a5d0 2024-09-16T09:02:14.455 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2272695765 0 0) 0x7f43281a89e0 con 0x7f4328101a70 2024-09-16T09:02:14.455 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 >> v1:172.21.15.160:6790/0 conn(0x7f432810e1a0 legacy=0x7f43281a5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:14.455 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 >> v1:172.21.15.169:6789/0 conn(0x7f432810a5d0 legacy=0x7f43281a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:14.455 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43281aae80 con 0x7f4328101a70 2024-09-16T09:02:14.455 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f432ed17640 1 -- 172.21.15.160:0/2391402354 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f43281a9e10 con 0x7f4328101a70 2024-09-16T09:02:14.455 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f431c003a60 con 0x7f4328101a70 2024-09-16T09:02:14.455 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f431c005ba0 con 0x7f4328101a70 2024-09-16T09:02:14.455 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.450+0000 7f432ed17640 1 -- 172.21.15.160:0/2391402354 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f43281aa450 con 0x7f4328101a70 2024-09-16T09:02:14.455 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.451+0000 7f432ed17640 1 -- 172.21.15.160:0/2391402354 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4328102d10 con 0x7f4328101a70 2024-09-16T09:02:14.456 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.452+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7f431c0035f0 con 0x7f4328101a70 2024-09-16T09:02:14.456 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.453+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(748..748 src has 1..748) ==== 7814+0+0 (unknown 250817019 0 0) 0x7f431c099a30 con 0x7f4328101a70 2024-09-16T09:02:14.458 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.457+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f431c0649d0 con 0x7f4328101a70 2024-09-16T09:02:14.635 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:14.634+0000 7f432ed17640 1 -- 172.21.15.160:0/2391402354 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "100"} v 0) -- 0x7f4328113f30 con 0x7f4328101a70 2024-09-16T09:02:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-mon[30137]: pgmap v1170: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3522397961' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "100"}]': finished 2024-09-16T09:02:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-mon[30137]: osdmap e747: 8 total, 8 up, 8 in 2024-09-16T09:02:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3522397961' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "100"} : dispatch 2024-09-16T09:02:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-mon[30137]: pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' no longer out of quota; removing NO_QUOTA flag 2024-09-16T09:02:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-mon[30137]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-16T09:02:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3522397961' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "100"}]': finished 2024-09-16T09:02:14.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-mon[30137]: osdmap e748: 8 total, 8 up, 8 in 2024-09-16T09:02:14.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-mon[40028]: pgmap v1170: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:14.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3522397961' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "100"}]': finished 2024-09-16T09:02:14.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-mon[40028]: osdmap e747: 8 total, 8 up, 8 in 2024-09-16T09:02:14.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3522397961' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "100"} : dispatch 2024-09-16T09:02:14.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-mon[40028]: pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' no longer out of quota; removing NO_QUOTA flag 2024-09-16T09:02:14.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-mon[40028]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-16T09:02:14.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3522397961' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "100"}]': finished 2024-09-16T09:02:14.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-mon[40028]: osdmap e748: 8 total, 8 up, 8 in 2024-09-16T09:02:14.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:14 smithi169 ceph-mon[33465]: pgmap v1170: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:14.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:14 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3522397961' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "100"}]': finished 2024-09-16T09:02:14.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:14 smithi169 ceph-mon[33465]: osdmap e747: 8 total, 8 up, 8 in 2024-09-16T09:02:14.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:14 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3522397961' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "100"} : dispatch 2024-09-16T09:02:14.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:14 smithi169 ceph-mon[33465]: pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' no longer out of quota; removing NO_QUOTA flag 2024-09-16T09:02:14.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:14 smithi169 ceph-mon[33465]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-16T09:02:14.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:14 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3522397961' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "100"}]': finished 2024-09-16T09:02:14.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:14 smithi169 ceph-mon[33465]: osdmap e748: 8 total, 8 up, 8 in 2024-09-16T09:02:15.385 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:02:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:02:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:02:15.389 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:15.388+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da v749) ==== 221+0+0 (unknown 2807663058 0 0) 0x7f431c069910 con 0x7f4328101a70 2024-09-16T09:02:15.499 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:15.497+0000 7f432ed17640 1 -- 172.21.15.160:0/2391402354 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "100"} v 0) -- 0x7f43281aa930 con 0x7f4328101a70 2024-09-16T09:02:15.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:15 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2391402354' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "100"} : dispatch 2024-09-16T09:02:15.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:15 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2391402354' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "100"} : dispatch 2024-09-16T09:02:15.850 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:02:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:02:15.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:15 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2391402354' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "100"} : dispatch 2024-09-16T09:02:16.391 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:16.390+0000 7f4325ffb640 1 -- 172.21.15.160:0/2391402354 <== mon.0 v1:172.21.15.160:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da v750) ==== 221+0+0 (unknown 574917888 0 0) 0x7f431c05c0d0 con 0x7f4328101a70 2024-09-16T09:02:16.392 INFO:tasks.workunit.client.0.smithi160.stderr:set-quota max_bytes = 100 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da 2024-09-16T09:02:16.395 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:16.393+0000 7f432ed17640 1 -- 172.21.15.160:0/2391402354 >> v1:172.21.15.160:6800/2859204856 conn(0x7f431807bd00 legacy=0x7f431807e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:16.395 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:16.393+0000 7f432ed17640 1 -- 172.21.15.160:0/2391402354 >> v1:172.21.15.160:6789/0 conn(0x7f4328101a70 legacy=0x7f432810d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:16.395 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:16.393+0000 7f432ed17640 1 -- 172.21.15.160:0/2391402354 shutdown_connections 2024-09-16T09:02:16.395 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:16.393+0000 7f432ed17640 1 -- 172.21.15.160:0/2391402354 >> 172.21.15.160:0/2391402354 conn(0x7f43280fd470 msgr2=0x7f432806b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:16.395 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:16.394+0000 7f432ed17640 1 -- 172.21.15.160:0/2391402354 shutdown_connections 2024-09-16T09:02:16.395 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:16.394+0000 7f432ed17640 1 -- 172.21.15.160:0/2391402354 wait complete. 2024-09-16T09:02:16.408 INFO:tasks.workunit.client.0.smithi160.stderr:+ sleep 30 2024-09-16T09:02:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:16 smithi160 ceph-mon[30137]: pgmap v1173: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2391402354' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "100"}]': finished 2024-09-16T09:02:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:16 smithi160 ceph-mon[30137]: osdmap e749: 8 total, 8 up, 8 in 2024-09-16T09:02:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:16 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2391402354' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "100"} : dispatch 2024-09-16T09:02:16.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:16 smithi160 ceph-mon[40028]: pgmap v1173: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2391402354' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "100"}]': finished 2024-09-16T09:02:16.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:16 smithi160 ceph-mon[40028]: osdmap e749: 8 total, 8 up, 8 in 2024-09-16T09:02:16.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:16 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2391402354' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "100"} : dispatch 2024-09-16T09:02:16.699 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:16 smithi169 ceph-mon[33465]: pgmap v1173: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2391402354' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "100"}]': finished 2024-09-16T09:02:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:16 smithi169 ceph-mon[33465]: osdmap e749: 8 total, 8 up, 8 in 2024-09-16T09:02:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:16 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2391402354' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "100"} : dispatch 2024-09-16T09:02:16.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:17.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:17 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2391402354' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "100"}]': finished 2024-09-16T09:02:17.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:17 smithi160 ceph-mon[30137]: osdmap e750: 8 total, 8 up, 8 in 2024-09-16T09:02:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:17 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2391402354' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "100"}]': finished 2024-09-16T09:02:17.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:17 smithi160 ceph-mon[40028]: osdmap e750: 8 total, 8 up, 8 in 2024-09-16T09:02:17.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:17 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2391402354' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "100"}]': finished 2024-09-16T09:02:17.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:17 smithi169 ceph-mon[33465]: osdmap e750: 8 total, 8 up, 8 in 2024-09-16T09:02:18.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:18 smithi160 ceph-mon[30137]: pgmap v1176: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:18.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:18 smithi160 ceph-mon[40028]: pgmap v1176: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:18.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:18 smithi169 ceph-mon[33465]: pgmap v1176: 176 pgs: 176 active+clean; 708 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:19.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:19 smithi160 ceph-mon[30137]: pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' is full (reached quota's max_bytes: 100 B) 2024-09-16T09:02:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:19 smithi160 ceph-mon[30137]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-16T09:02:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:19 smithi160 ceph-mon[30137]: osdmap e751: 8 total, 8 up, 8 in 2024-09-16T09:02:19.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:19 smithi160 ceph-mon[40028]: pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' is full (reached quota's max_bytes: 100 B) 2024-09-16T09:02:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:19 smithi160 ceph-mon[40028]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-16T09:02:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:19 smithi160 ceph-mon[40028]: osdmap e751: 8 total, 8 up, 8 in 2024-09-16T09:02:19.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:19.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:19 smithi169 ceph-mon[33465]: pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' is full (reached quota's max_bytes: 100 B) 2024-09-16T09:02:19.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:19 smithi169 ceph-mon[33465]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-16T09:02:19.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:19 smithi169 ceph-mon[33465]: osdmap e751: 8 total, 8 up, 8 in 2024-09-16T09:02:19.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:20.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:20 smithi160 ceph-mon[30137]: pgmap v1178: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1016 B/s rd, 5.2 KiB/s wr, 1 op/s 2024-09-16T09:02:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:20 smithi160 ceph-mon[40028]: pgmap v1178: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1016 B/s rd, 5.2 KiB/s wr, 1 op/s 2024-09-16T09:02:20.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:20 smithi169 ceph-mon[33465]: pgmap v1178: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1016 B/s rd, 5.2 KiB/s wr, 1 op/s 2024-09-16T09:02:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:22 smithi160 ceph-mon[40028]: pgmap v1179: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-16T09:02:22.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:22 smithi160 ceph-mon[30137]: pgmap v1179: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-16T09:02:22.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:22 smithi169 ceph-mon[33465]: pgmap v1179: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-16T09:02:24.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:24 smithi160 ceph-mon[30137]: pgmap v1180: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 643 B/s rd, 3.3 KiB/s wr, 0 op/s 2024-09-16T09:02:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:24 smithi160 ceph-mon[40028]: pgmap v1180: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 643 B/s rd, 3.3 KiB/s wr, 0 op/s 2024-09-16T09:02:24.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:24 smithi169 ceph-mon[33465]: pgmap v1180: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 643 B/s rd, 3.3 KiB/s wr, 0 op/s 2024-09-16T09:02:25.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:02:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:02:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:02:26.100 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:02:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:02:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:26 smithi160 ceph-mon[30137]: pgmap v1181: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-09-16T09:02:26.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:26 smithi160 ceph-mon[40028]: pgmap v1181: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-09-16T09:02:26.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:26.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:26 smithi169 ceph-mon[33465]: pgmap v1181: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-09-16T09:02:26.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:28.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:28 smithi160 ceph-mon[30137]: pgmap v1182: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-16T09:02:28.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:28 smithi160 ceph-mon[40028]: pgmap v1182: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-16T09:02:28.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:28 smithi169 ceph-mon[33465]: pgmap v1182: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-16T09:02:30.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:30 smithi160 ceph-mon[30137]: pgmap v1183: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-09-16T09:02:30.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:30 smithi160 ceph-mon[40028]: pgmap v1183: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-09-16T09:02:30.852 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:30 smithi169 ceph-mon[33465]: pgmap v1183: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-09-16T09:02:32.697 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:32 smithi160 ceph-mon[30137]: pgmap v1184: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:32.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:32 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:02:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:32 smithi160 ceph-mon[40028]: pgmap v1184: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:32.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:32 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:02:32.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:32 smithi169 ceph-mon[33465]: pgmap v1184: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:32.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:32 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:02:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:33 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:02:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:33 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:02:33.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:33 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:02:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:33 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:02:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:33 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:02:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:33 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:02:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:33 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:02:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:33 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:02:33.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:33 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:02:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:34 smithi169 ceph-mon[33465]: pgmap v1185: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:34.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:34.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:34 smithi160 ceph-mon[30137]: pgmap v1185: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:34.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:34.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:34 smithi160 ceph-mon[40028]: pgmap v1185: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:34.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:35.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:02:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:02:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:02:36.100 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:02:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:02:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:36 smithi169 ceph-mon[33465]: pgmap v1186: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:36.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:36 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:36.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:36 smithi160 ceph-mon[30137]: pgmap v1186: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:36.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:36 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:36.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:36 smithi160 ceph-mon[40028]: pgmap v1186: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:36.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:36 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:38.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:38 smithi169 ceph-mon[33465]: pgmap v1187: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-16T09:02:38.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:38 smithi160 ceph-mon[30137]: pgmap v1187: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-16T09:02:38.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:38 smithi160 ceph-mon[40028]: pgmap v1187: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-16T09:02:40.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:40 smithi169 ceph-mon[33465]: pgmap v1188: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:40 smithi160 ceph-mon[30137]: pgmap v1188: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:40.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:40 smithi160 ceph-mon[40028]: pgmap v1188: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:42.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:42 smithi169 ceph-mon[33465]: pgmap v1189: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:42.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:42 smithi160 ceph-mon[30137]: pgmap v1189: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:42.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:42 smithi160 ceph-mon[40028]: pgmap v1189: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:44.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:44 smithi169 ceph-mon[33465]: pgmap v1190: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:44 smithi160 ceph-mon[30137]: pgmap v1190: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:44 smithi160 ceph-mon[40028]: pgmap v1190: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:45.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:02:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:02:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:02:46.100 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:02:45 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:02:46.410 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=146089 2024-09-16T09:02:46.410 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph osd pool set-quota 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da max_bytes 0 2024-09-16T09:02:46.411 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da put two /etc/passwd 2024-09-16T09:02:46.505 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.504+0000 7f8365201640 1 -- 172.21.15.160:0/1805565694 >> v1:172.21.15.160:6789/0 conn(0x7f8360119c90 legacy=0x7f836011c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:46.505 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.504+0000 7f8365201640 1 -- 172.21.15.160:0/1805565694 shutdown_connections 2024-09-16T09:02:46.506 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.504+0000 7f8365201640 1 -- 172.21.15.160:0/1805565694 >> 172.21.15.160:0/1805565694 conn(0x7f836006c2a0 msgr2=0x7f836006c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:46.506 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.504+0000 7f8365201640 1 -- 172.21.15.160:0/1805565694 shutdown_connections 2024-09-16T09:02:46.506 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.504+0000 7f8365201640 1 -- 172.21.15.160:0/1805565694 wait complete. 2024-09-16T09:02:46.506 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.505+0000 7f8365201640 1 Processor -- start 2024-09-16T09:02:46.509 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.505+0000 7f8365201640 1 -- start start 2024-09-16T09:02:46.509 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.505+0000 7f8365201640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f83601b2220 con 0x7f8360119c90 2024-09-16T09:02:46.509 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.505+0000 7f8365201640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f83601b3420 con 0x7f836010d560 2024-09-16T09:02:46.509 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.505+0000 7f8365201640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f83601b4620 con 0x7f83601160c0 2024-09-16T09:02:46.509 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.505+0000 7f835f59e640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f8360119c90 0x7f83601b08f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:43412/0 (socket says 172.21.15.160:43412) 2024-09-16T09:02:46.509 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f835f59e640 1 -- 172.21.15.160:0/1438666983 learned_addr learned my addr 172.21.15.160:0/1438666983 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T09:02:46.509 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4062110418 0 0) 0x7f83601b2220 con 0x7f8360119c90 2024-09-16T09:02:46.509 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f834c003160 con 0x7f8360119c90 2024-09-16T09:02:46.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2843789532 0 0) 0x7f83601b4620 con 0x7f83601160c0 2024-09-16T09:02:46.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83601b2220 con 0x7f83601160c0 2024-09-16T09:02:46.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1984457403 0 0) 0x7f83601b3420 con 0x7f836010d560 2024-09-16T09:02:46.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83601b4620 con 0x7f836010d560 2024-09-16T09:02:46.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4277422426 0 0) 0x7f83601b2220 con 0x7f83601160c0 2024-09-16T09:02:46.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f83601b3420 con 0x7f83601160c0 2024-09-16T09:02:46.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1494957616 0 0) 0x7f834c003160 con 0x7f8360119c90 2024-09-16T09:02:46.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f83601b2220 con 0x7f8360119c90 2024-09-16T09:02:46.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3935339895 0 0) 0x7f83601b4620 con 0x7f836010d560 2024-09-16T09:02:46.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f834c003160 con 0x7f836010d560 2024-09-16T09:02:46.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f83480018f0 con 0x7f83601160c0 2024-09-16T09:02:46.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f8358001dd0 con 0x7f8360119c90 2024-09-16T09:02:46.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f83500010a0 con 0x7f836010d560 2024-09-16T09:02:46.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3326365435 0 0) 0x7f83601b3420 con 0x7f83601160c0 2024-09-16T09:02:46.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 >> v1:172.21.15.169:6789/0 conn(0x7f836010d560 legacy=0x7f8360119380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:46.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 >> v1:172.21.15.160:6789/0 conn(0x7f8360119c90 legacy=0x7f83601b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:46.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83601b58c0 con 0x7f83601160c0 2024-09-16T09:02:46.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8365201640 1 -- 172.21.15.160:0/1438666983 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f83601b2450 con 0x7f83601160c0 2024-09-16T09:02:46.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8365201640 1 -- 172.21.15.160:0/1438666983 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f83601b2a90 con 0x7f83601160c0 2024-09-16T09:02:46.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f8348001c50 con 0x7f83601160c0 2024-09-16T09:02:46.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.506+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f8348004a40 con 0x7f83601160c0 2024-09-16T09:02:46.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.507+0000 7f8365201640 1 -- 172.21.15.160:0/1438666983 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83400050b0 con 0x7f83601160c0 2024-09-16T09:02:46.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.508+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7f83480043c0 con 0x7f83601160c0 2024-09-16T09:02:46.512 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.509+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(751..751 src has 1..751) ==== 7814+0+0 (unknown 3648628192 0 0) 0x7f8348098740 con 0x7f83601160c0 2024-09-16T09:02:46.512 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.509+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=752}) -- 0x7f83601b3420 con 0x7f83601160c0 2024-09-16T09:02:46.513 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.513+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f8348063740 con 0x7f83601160c0 2024-09-16T09:02:46.694 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:46.694+0000 7f8365201640 1 -- 172.21.15.160:0/1438666983 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} v 0) -- 0x7f83400053a0 con 0x7f83601160c0 2024-09-16T09:02:46.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:46 smithi169 ceph-mon[33465]: pgmap v1191: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:46.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:46 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:46.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:46 smithi160 ceph-mon[30137]: pgmap v1191: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:46.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:46 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:46.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:46 smithi160 ceph-mon[40028]: pgmap v1191: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:46.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:46 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:47.530 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:47.529+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.2 v1:172.21.15.160:6790/0 10 ==== osd_map(752..752 src has 1..752) ==== 628+0+0 (unknown 2938542863 0 0) 0x7f8348096540 con 0x7f83601160c0 2024-09-16T09:02:47.530 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:47.529+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=753}) -- 0x7f83601b4620 con 0x7f83601160c0 2024-09-16T09:02:47.533 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:47.532+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.2 v1:172.21.15.160:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da v752) ==== 217+0+0 (unknown 3275957722 0 0) 0x7f8348068680 con 0x7f83601160c0 2024-09-16T09:02:47.644 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:47.644+0000 7f8365201640 1 -- 172.21.15.160:0/1438666983 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} v 0) -- 0x7f8340005c70 con 0x7f83601160c0 2024-09-16T09:02:47.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:47 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1438666983' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:02:47.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:47 smithi169 ceph-mon[33465]: from='client.57416 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:02:47.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:47 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1438666983' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:02:47.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:47 smithi160 ceph-mon[30137]: from='client.57416 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:02:47.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:47 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1438666983' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:02:47.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:47 smithi160 ceph-mon[40028]: from='client.57416 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:02:48.529 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.529+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.2 v1:172.21.15.160:6790/0 12 ==== osd_map(753..753 src has 1..753) ==== 628+0+0 (unknown 1409710419 0 0) 0x7f834805e3d0 con 0x7f83601160c0 2024-09-16T09:02:48.530 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.529+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=754}) -- 0x7f834c088ed0 con 0x7f83601160c0 2024-09-16T09:02:48.535 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.534+0000 7f8357fff640 1 -- 172.21.15.160:0/1438666983 <== mon.2 v1:172.21.15.160:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da v753) ==== 217+0+0 (unknown 303573012 0 0) 0x7f83480681f0 con 0x7f83601160c0 2024-09-16T09:02:48.535 INFO:tasks.workunit.client.0.smithi160.stderr:set-quota max_bytes = 0 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da 2024-09-16T09:02:48.539 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.538+0000 7f8365201640 1 -- 172.21.15.160:0/1438666983 >> v1:172.21.15.160:6800/2859204856 conn(0x7f834c07bce0 legacy=0x7f834c07e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:48.539 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.538+0000 7f8365201640 1 -- 172.21.15.160:0/1438666983 >> v1:172.21.15.160:6790/0 conn(0x7f83601160c0 legacy=0x7f83601ad0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:48.539 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.538+0000 7f8365201640 1 -- 172.21.15.160:0/1438666983 shutdown_connections 2024-09-16T09:02:48.539 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.538+0000 7f8365201640 1 -- 172.21.15.160:0/1438666983 >> 172.21.15.160:0/1438666983 conn(0x7f836006c2a0 msgr2=0x7f8360111590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:48.539 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.538+0000 7f8365201640 1 -- 172.21.15.160:0/1438666983 shutdown_connections 2024-09-16T09:02:48.539 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.538+0000 7f8365201640 1 -- 172.21.15.160:0/1438666983 wait complete. 2024-09-16T09:02:48.550 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph osd pool set-quota 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da max_objects 0 2024-09-16T09:02:48.639 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.636+0000 7fc4085da640 1 -- 172.21.15.160:0/2601501851 >> v1:172.21.15.160:6790/0 conn(0x7fc40010b6c0 legacy=0x7fc40010dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:48.640 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.636+0000 7fc4085da640 1 -- 172.21.15.160:0/2601501851 shutdown_connections 2024-09-16T09:02:48.640 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.636+0000 7fc4085da640 1 -- 172.21.15.160:0/2601501851 >> 172.21.15.160:0/2601501851 conn(0x7fc4000fe5c0 msgr2=0x7fc4001009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:48.640 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.636+0000 7fc4085da640 1 -- 172.21.15.160:0/2601501851 shutdown_connections 2024-09-16T09:02:48.640 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.636+0000 7fc4085da640 1 -- 172.21.15.160:0/2601501851 wait complete. 2024-09-16T09:02:48.640 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.637+0000 7fc4085da640 1 Processor -- start 2024-09-16T09:02:48.640 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.637+0000 7fc4085da640 1 -- start start 2024-09-16T09:02:48.640 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.637+0000 7fc4085da640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc4001a77f0 con 0x7fc40010f290 2024-09-16T09:02:48.640 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.637+0000 7fc4085da640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc4001a89f0 con 0x7fc400102b60 2024-09-16T09:02:48.640 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.637+0000 7fc4085da640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc4001a9bf0 con 0x7fc40010b6c0 2024-09-16T09:02:48.640 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.637+0000 7fc406b77640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fc40010f290 0x7fc4001a5ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:51344/0 (socket says 172.21.15.160:51344) 2024-09-16T09:02:48.641 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.637+0000 7fc406b77640 1 -- 172.21.15.160:0/2413137176 learned_addr learned my addr 172.21.15.160:0/2413137176 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T09:02:48.641 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.637+0000 7fc405b75640 1 --1- 172.21.15.160:0/2413137176 >> v1:172.21.15.160:6790/0 conn(0x7fc40010b6c0 0x7fc4001a26c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:43410/0 (socket says 172.21.15.160:43410) 2024-09-16T09:02:48.641 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.637+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3574631340 0 0) 0x7fc4001a77f0 con 0x7fc40010f290 2024-09-16T09:02:48.641 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.637+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3ec003520 con 0x7fc40010f290 2024-09-16T09:02:48.641 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.637+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 239353674 0 0) 0x7fc4001a9bf0 con 0x7fc40010b6c0 2024-09-16T09:02:48.641 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.637+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4001a77f0 con 0x7fc40010b6c0 2024-09-16T09:02:48.641 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.637+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2243106646 0 0) 0x7fc4001a77f0 con 0x7fc40010b6c0 2024-09-16T09:02:48.641 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.637+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc4001a9bf0 con 0x7fc40010b6c0 2024-09-16T09:02:48.641 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.637+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fc3f0001650 con 0x7fc40010b6c0 2024-09-16T09:02:48.641 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.638+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2217953301 0 0) 0x7fc3ec003520 con 0x7fc40010f290 2024-09-16T09:02:48.641 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.638+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc4001a77f0 con 0x7fc40010f290 2024-09-16T09:02:48.641 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.638+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 490271056 0 0) 0x7fc4001a89f0 con 0x7fc400102b60 2024-09-16T09:02:48.642 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.638+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3ec003520 con 0x7fc400102b60 2024-09-16T09:02:48.642 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.638+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fc3f8001690 con 0x7fc40010f290 2024-09-16T09:02:48.642 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.638+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1822435746 0 0) 0x7fc4001a9bf0 con 0x7fc40010b6c0 2024-09-16T09:02:48.642 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.638+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 >> v1:172.21.15.169:6789/0 conn(0x7fc400102b60 legacy=0x7fc40010e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:48.642 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.638+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 >> v1:172.21.15.160:6789/0 conn(0x7fc40010f290 legacy=0x7fc4001a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:48.642 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.638+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4001aae90 con 0x7fc40010b6c0 2024-09-16T09:02:48.642 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.638+0000 7fc4085da640 1 -- 172.21.15.160:0/2413137176 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc4001a9e20 con 0x7fc40010b6c0 2024-09-16T09:02:48.642 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.638+0000 7fc4085da640 1 -- 172.21.15.160:0/2413137176 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fc4001aa460 con 0x7fc40010b6c0 2024-09-16T09:02:48.643 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.638+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7fc3f0001ba0 con 0x7fc40010b6c0 2024-09-16T09:02:48.643 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.638+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7fc3f0004d30 con 0x7fc40010b6c0 2024-09-16T09:02:48.643 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.639+0000 7fc4085da640 1 -- 172.21.15.160:0/2413137176 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e4005080 con 0x7fc40010b6c0 2024-09-16T09:02:48.643 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.639+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7fc3f00035f0 con 0x7fc40010b6c0 2024-09-16T09:02:48.643 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.640+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(753..753 src has 1..753) ==== 7814+0+0 (unknown 3552419289 0 0) 0x7fc3f0099a60 con 0x7fc40010b6c0 2024-09-16T09:02:48.644 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.640+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=754}) -- 0x7fc4001a9bf0 con 0x7fc40010b6c0 2024-09-16T09:02:48.644 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.644+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7fc3f0064a60 con 0x7fc40010b6c0 2024-09-16T09:02:48.823 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:48.823+0000 7fc4085da640 1 -- 172.21.15.160:0/2413137176 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} v 0) -- 0x7fc3e4005370 con 0x7fc40010b6c0 2024-09-16T09:02:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:48 smithi169 ceph-mon[33465]: pgmap v1192: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:48 smithi169 ceph-mon[33465]: from='client.57416 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"}]': finished 2024-09-16T09:02:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:48 smithi169 ceph-mon[33465]: osdmap e752: 8 total, 8 up, 8 in 2024-09-16T09:02:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:48 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1438666983' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:02:48.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:48 smithi169 ceph-mon[33465]: from='client.57416 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:02:48.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:48 smithi160 ceph-mon[30137]: pgmap v1192: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:48.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:48 smithi160 ceph-mon[30137]: from='client.57416 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"}]': finished 2024-09-16T09:02:48.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:48 smithi160 ceph-mon[30137]: osdmap e752: 8 total, 8 up, 8 in 2024-09-16T09:02:48.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:48 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1438666983' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:02:48.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:48 smithi160 ceph-mon[30137]: from='client.57416 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:02:48.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:48 smithi160 ceph-mon[40028]: pgmap v1192: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:02:48.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:48 smithi160 ceph-mon[40028]: from='client.57416 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"}]': finished 2024-09-16T09:02:48.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:48 smithi160 ceph-mon[40028]: osdmap e752: 8 total, 8 up, 8 in 2024-09-16T09:02:48.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:48 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1438666983' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:02:48.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:48 smithi160 ceph-mon[40028]: from='client.57416 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:02:49.316 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:49.316+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.2 v1:172.21.15.160:6790/0 10 ==== osd_map(754..754 src has 1..754) ==== 628+0+0 (unknown 4003914608 0 0) 0x7fc3f0097860 con 0x7fc40010b6c0 2024-09-16T09:02:49.316 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:49.316+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=755}) -- 0x7fc4001a89f0 con 0x7fc40010b6c0 2024-09-16T09:02:49.320 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:49.320+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.2 v1:172.21.15.160:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da v754) ==== 221+0+0 (unknown 1891392220 0 0) 0x7fc3f00699a0 con 0x7fc40010b6c0 2024-09-16T09:02:49.438 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:49.437+0000 7fc4085da640 1 -- 172.21.15.160:0/2413137176 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} v 0) -- 0x7fc3e4001090 con 0x7fc40010b6c0 2024-09-16T09:02:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:49 smithi169 ceph-mon[33465]: from='client.57416 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"}]': finished 2024-09-16T09:02:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:49 smithi169 ceph-mon[33465]: osdmap e753: 8 total, 8 up, 8 in 2024-09-16T09:02:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2413137176' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:02:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:49 smithi169 ceph-mon[33465]: from='client.57422 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:02:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:49 smithi169 ceph-mon[33465]: pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' no longer out of quota; removing NO_QUOTA flag 2024-09-16T09:02:49.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:49 smithi169 ceph-mon[33465]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-16T09:02:49.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:49 smithi169 ceph-mon[33465]: from='client.57422 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"}]': finished 2024-09-16T09:02:49.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:49 smithi169 ceph-mon[33465]: osdmap e754: 8 total, 8 up, 8 in 2024-09-16T09:02:49.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:02:49.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:49.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:49 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2413137176' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:02:49.851 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:49 smithi169 ceph-mon[33465]: from='client.57422 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:02:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[30137]: from='client.57416 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"}]': finished 2024-09-16T09:02:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[30137]: osdmap e753: 8 total, 8 up, 8 in 2024-09-16T09:02:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2413137176' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:02:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[30137]: from='client.57422 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:02:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[30137]: pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' no longer out of quota; removing NO_QUOTA flag 2024-09-16T09:02:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[30137]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-16T09:02:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[30137]: from='client.57422 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"}]': finished 2024-09-16T09:02:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[30137]: osdmap e754: 8 total, 8 up, 8 in 2024-09-16T09:02:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:02:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:49.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2413137176' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:02:49.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[30137]: from='client.57422 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:02:49.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[40028]: from='client.57416 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"}]': finished 2024-09-16T09:02:49.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[40028]: osdmap e753: 8 total, 8 up, 8 in 2024-09-16T09:02:49.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2413137176' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:02:49.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[40028]: from='client.57422 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:02:49.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[40028]: pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' no longer out of quota; removing NO_QUOTA flag 2024-09-16T09:02:49.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[40028]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-16T09:02:49.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[40028]: from='client.57422 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"}]': finished 2024-09-16T09:02:49.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[40028]: osdmap e754: 8 total, 8 up, 8 in 2024-09-16T09:02:49.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:02:49.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:49.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2413137176' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:02:49.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:49 smithi160 ceph-mon[40028]: from='client.57422 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:02:50.364 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.364+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.2 v1:172.21.15.160:6790/0 12 ==== osd_map(755..755 src has 1..755) ==== 628+0+0 (unknown 2117818254 0 0) 0x7fc3f005f6f0 con 0x7fc40010b6c0 2024-09-16T09:02:50.368 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.367+0000 7fc3f77fe640 1 -- 172.21.15.160:0/2413137176 <== mon.2 v1:172.21.15.160:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da v755) ==== 221+0+0 (unknown 4042033981 0 0) 0x7fc3f0069510 con 0x7fc40010b6c0 2024-09-16T09:02:50.368 INFO:tasks.workunit.client.0.smithi160.stderr:set-quota max_objects = 0 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da 2024-09-16T09:02:50.372 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.372+0000 7fc4085da640 1 -- 172.21.15.160:0/2413137176 >> v1:172.21.15.160:6800/2859204856 conn(0x7fc3ec07baf0 legacy=0x7fc3ec07dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:50.373 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.372+0000 7fc4085da640 1 -- 172.21.15.160:0/2413137176 >> v1:172.21.15.160:6790/0 conn(0x7fc40010b6c0 legacy=0x7fc4001a26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:50.373 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.372+0000 7fc4085da640 1 -- 172.21.15.160:0/2413137176 shutdown_connections 2024-09-16T09:02:50.373 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.372+0000 7fc4085da640 1 -- 172.21.15.160:0/2413137176 >> 172.21.15.160:0/2413137176 conn(0x7fc4000fe5c0 msgr2=0x7fc400101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:50.373 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.372+0000 7fc4085da640 1 -- 172.21.15.160:0/2413137176 shutdown_connections 2024-09-16T09:02:50.373 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.372+0000 7fc4085da640 1 -- 172.21.15.160:0/2413137176 wait complete. 2024-09-16T09:02:50.384 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 146089 2024-09-16T09:02:50.384 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 0 -ne 0 ']' 2024-09-16T09:02:50.384 INFO:tasks.workunit.client.0.smithi160.stderr:+ true 2024-09-16T09:02:50.384 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da put three /etc/passwd 2024-09-16T09:02:50.422 INFO:tasks.workunit.client.0.smithi160.stderr:++ uuidgen 2024-09-16T09:02:50.423 INFO:tasks.workunit.client.0.smithi160.stderr:+ pp=b3210054-d6e7-4974-b854-ef9506616c26 2024-09-16T09:02:50.423 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph osd pool create b3210054-d6e7-4974-b854-ef9506616c26 12 2024-09-16T09:02:50.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.507+0000 7f55aa92c640 1 -- 172.21.15.160:0/3591479524 >> v1:172.21.15.160:6790/0 conn(0x7f55a410a5c0 legacy=0x7f55a410c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:50.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.507+0000 7f55aa92c640 1 -- 172.21.15.160:0/3591479524 shutdown_connections 2024-09-16T09:02:50.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.507+0000 7f55aa92c640 1 -- 172.21.15.160:0/3591479524 >> 172.21.15.160:0/3591479524 conn(0x7f55a40773e0 msgr2=0x7f55a40777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:50.510 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.507+0000 7f55aa92c640 1 -- 172.21.15.160:0/3591479524 shutdown_connections 2024-09-16T09:02:50.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.507+0000 7f55aa92c640 1 -- 172.21.15.160:0/3591479524 wait complete. 2024-09-16T09:02:50.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55aa92c640 1 Processor -- start 2024-09-16T09:02:50.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55aa92c640 1 -- start start 2024-09-16T09:02:50.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55aa92c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f55a41a7660 con 0x7f55a4101cb0 2024-09-16T09:02:50.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55aa92c640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f55a41a8860 con 0x7f55a410a5c0 2024-09-16T09:02:50.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55aa92c640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f55a41a9a60 con 0x7f55a410e190 2024-09-16T09:02:50.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55a3fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f55a4101cb0 0x7f55a410d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:51376/0 (socket says 172.21.15.160:51376) 2024-09-16T09:02:50.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55a3fff640 1 -- 172.21.15.160:0/2206000615 learned_addr learned my addr 172.21.15.160:0/2206000615 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T09:02:50.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 920131258 0 0) 0x7f55a41a7660 con 0x7f55a4101cb0 2024-09-16T09:02:50.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5590003520 con 0x7f55a4101cb0 2024-09-16T09:02:50.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2439893138 0 0) 0x7f55a41a9a60 con 0x7f55a410e190 2024-09-16T09:02:50.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55a41a7660 con 0x7f55a410e190 2024-09-16T09:02:50.511 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3564740822 0 0) 0x7f55a41a8860 con 0x7f55a410a5c0 2024-09-16T09:02:50.512 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55a41a9a60 con 0x7f55a410a5c0 2024-09-16T09:02:50.512 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1775736573 0 0) 0x7f5590003520 con 0x7f55a4101cb0 2024-09-16T09:02:50.512 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f55a41a8860 con 0x7f55a4101cb0 2024-09-16T09:02:50.512 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f5594001a10 con 0x7f55a4101cb0 2024-09-16T09:02:50.512 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1244796684 0 0) 0x7f55a41a7660 con 0x7f55a410e190 2024-09-16T09:02:50.512 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5590003520 con 0x7f55a410e190 2024-09-16T09:02:50.512 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.508+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f559c001b20 con 0x7f55a410e190 2024-09-16T09:02:50.512 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.509+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1227531668 0 0) 0x7f55a41a9a60 con 0x7f55a410a5c0 2024-09-16T09:02:50.512 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.509+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f55a41a7660 con 0x7f55a410a5c0 2024-09-16T09:02:50.512 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.509+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f558c001a70 con 0x7f55a410a5c0 2024-09-16T09:02:50.512 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.509+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1042697272 0 0) 0x7f55a41a8860 con 0x7f55a4101cb0 2024-09-16T09:02:50.512 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.509+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 >> v1:172.21.15.160:6790/0 conn(0x7f55a410e190 legacy=0x7f55a41a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:50.513 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.509+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 >> v1:172.21.15.169:6789/0 conn(0x7f55a410a5c0 legacy=0x7f55a41a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:50.513 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.509+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a41aad00 con 0x7f55a4101cb0 2024-09-16T09:02:50.513 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.509+0000 7f55aa92c640 1 -- 172.21.15.160:0/2206000615 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f55a41a7830 con 0x7f55a4101cb0 2024-09-16T09:02:50.513 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.509+0000 7f55aa92c640 1 -- 172.21.15.160:0/2206000615 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f55a41a7e70 con 0x7f55a4101cb0 2024-09-16T09:02:50.513 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.509+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f5594001c90 con 0x7f55a4101cb0 2024-09-16T09:02:50.513 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.509+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f5594005c00 con 0x7f55a4101cb0 2024-09-16T09:02:50.513 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.510+0000 7f55aa92c640 1 -- 172.21.15.160:0/2206000615 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55840050b0 con 0x7f55a4101cb0 2024-09-16T09:02:50.513 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.510+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7f55940038a0 con 0x7f55a4101cb0 2024-09-16T09:02:50.514 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.511+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(755..755 src has 1..755) ==== 7814+0+0 (unknown 3052058724 0 0) 0x7f5594099850 con 0x7f55a4101cb0 2024-09-16T09:02:50.515 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.515+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f55940647f0 con 0x7f55a4101cb0 2024-09-16T09:02:50.694 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:50.694+0000 7f55aa92c640 1 -- 172.21.15.160:0/2206000615 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pg_num": 12} v 0) -- 0x7f55840053a0 con 0x7f55a4101cb0 2024-09-16T09:02:50.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:50 smithi169 ceph-mon[33465]: pgmap v1196: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:02:50.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:50 smithi169 ceph-mon[33465]: from='client.57422 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"}]': finished 2024-09-16T09:02:50.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:50 smithi169 ceph-mon[33465]: osdmap e755: 8 total, 8 up, 8 in 2024-09-16T09:02:50.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:50 smithi160 ceph-mon[30137]: pgmap v1196: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:02:50.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:50 smithi160 ceph-mon[30137]: from='client.57422 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"}]': finished 2024-09-16T09:02:50.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:50 smithi160 ceph-mon[30137]: osdmap e755: 8 total, 8 up, 8 in 2024-09-16T09:02:50.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:50 smithi160 ceph-mon[40028]: pgmap v1196: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:02:50.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:50 smithi160 ceph-mon[40028]: from='client.57422 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"}]': finished 2024-09-16T09:02:50.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:50 smithi160 ceph-mon[40028]: osdmap e755: 8 total, 8 up, 8 in 2024-09-16T09:02:51.548 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.547+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pg_num": 12}]=0 pool 'b3210054-d6e7-4974-b854-ef9506616c26' created v756) ==== 176+0+0 (unknown 933743178 0 0) 0x7f5594069730 con 0x7f55a4101cb0 2024-09-16T09:02:51.665 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.664+0000 7f55aa92c640 1 -- 172.21.15.160:0/2206000615 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pg_num": 12} v 0) -- 0x7f55840058b0 con 0x7f55a4101cb0 2024-09-16T09:02:51.665 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.665+0000 7f55a1ffb640 1 -- 172.21.15.160:0/2206000615 <== mon.0 v1:172.21.15.160:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pg_num": 12}]=0 pool 'b3210054-d6e7-4974-b854-ef9506616c26' already exists v756) ==== 183+0+0 (unknown 4265107619 0 0) 0x7f55940975f0 con 0x7f55a4101cb0 2024-09-16T09:02:51.665 INFO:tasks.workunit.client.0.smithi160.stderr:pool 'b3210054-d6e7-4974-b854-ef9506616c26' already exists 2024-09-16T09:02:51.667 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.667+0000 7f55aa92c640 1 -- 172.21.15.160:0/2206000615 >> v1:172.21.15.160:6800/2859204856 conn(0x7f559007bbe0 legacy=0x7f559007e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:51.668 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.667+0000 7f55aa92c640 1 -- 172.21.15.160:0/2206000615 >> v1:172.21.15.160:6789/0 conn(0x7f55a4101cb0 legacy=0x7f55a410d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:51.668 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.667+0000 7f55aa92c640 1 -- 172.21.15.160:0/2206000615 shutdown_connections 2024-09-16T09:02:51.668 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.667+0000 7f55aa92c640 1 -- 172.21.15.160:0/2206000615 >> 172.21.15.160:0/2206000615 conn(0x7f55a40773e0 msgr2=0x7f55a4111620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:51.668 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.667+0000 7f55aa92c640 1 -- 172.21.15.160:0/2206000615 shutdown_connections 2024-09-16T09:02:51.668 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.667+0000 7f55aa92c640 1 -- 172.21.15.160:0/2206000615 wait complete. 2024-09-16T09:02:51.677 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph osd pool application enable b3210054-d6e7-4974-b854-ef9506616c26 rados 2024-09-16T09:02:51.758 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.757+0000 7f59eb06b640 1 -- 172.21.15.160:0/3083371 >> v1:172.21.15.160:6789/0 conn(0x7f59e410e190 legacy=0x7f59e4110630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:51.758 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.757+0000 7f59eb06b640 1 -- 172.21.15.160:0/3083371 shutdown_connections 2024-09-16T09:02:51.758 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.757+0000 7f59eb06b640 1 -- 172.21.15.160:0/3083371 >> 172.21.15.160:0/3083371 conn(0x7f59e40773e0 msgr2=0x7f59e40777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:51.758 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.757+0000 7f59eb06b640 1 -- 172.21.15.160:0/3083371 shutdown_connections 2024-09-16T09:02:51.759 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.757+0000 7f59eb06b640 1 -- 172.21.15.160:0/3083371 wait complete. 2024-09-16T09:02:51.759 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59eb06b640 1 Processor -- start 2024-09-16T09:02:51.759 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59eb06b640 1 -- start start 2024-09-16T09:02:51.759 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59eb06b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f59e41a7700 con 0x7f59e410e190 2024-09-16T09:02:51.759 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59eb06b640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f59e41a8900 con 0x7f59e410a5c0 2024-09-16T09:02:51.759 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59eb06b640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f59e41a9b00 con 0x7f59e4101cb0 2024-09-16T09:02:51.759 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59e8e07640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f59e4101cb0 0x7f59e410d850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:43490/0 (socket says 172.21.15.160:43490) 2024-09-16T09:02:51.759 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59e8e07640 1 -- 172.21.15.160:0/1631819600 learned_addr learned my addr 172.21.15.160:0/1631819600 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T09:02:51.761 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 760047333 0 0) 0x7f59e41a9b00 con 0x7f59e4101cb0 2024-09-16T09:02:51.762 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59d4003160 con 0x7f59e4101cb0 2024-09-16T09:02:51.762 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3959372978 0 0) 0x7f59e41a7700 con 0x7f59e410e190 2024-09-16T09:02:51.762 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59e41a9b00 con 0x7f59e410e190 2024-09-16T09:02:51.762 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2959993631 0 0) 0x7f59e41a8900 con 0x7f59e410a5c0 2024-09-16T09:02:51.762 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59e41a7700 con 0x7f59e410a5c0 2024-09-16T09:02:51.762 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3344105774 0 0) 0x7f59d4003160 con 0x7f59e4101cb0 2024-09-16T09:02:51.762 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f59e41a8900 con 0x7f59e4101cb0 2024-09-16T09:02:51.762 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f59d8001410 con 0x7f59e4101cb0 2024-09-16T09:02:51.762 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3497218147 0 0) 0x7f59e41a9b00 con 0x7f59e410e190 2024-09-16T09:02:51.762 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f59d4003160 con 0x7f59e410e190 2024-09-16T09:02:51.762 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.758+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f59dc001710 con 0x7f59e410e190 2024-09-16T09:02:51.763 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.759+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3551163912 0 0) 0x7f59e41a7700 con 0x7f59e410a5c0 2024-09-16T09:02:51.763 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.759+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f59e41a9b00 con 0x7f59e410a5c0 2024-09-16T09:02:51.763 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.759+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f59d0001830 con 0x7f59e410a5c0 2024-09-16T09:02:51.763 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.759+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 62955360 0 0) 0x7f59e41a8900 con 0x7f59e4101cb0 2024-09-16T09:02:51.763 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.759+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 >> v1:172.21.15.169:6789/0 conn(0x7f59e410a5c0 legacy=0x7f59e41a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:51.763 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.759+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 >> v1:172.21.15.160:6789/0 conn(0x7f59e410e190 legacy=0x7f59e41a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:51.763 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.759+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e41aada0 con 0x7f59e4101cb0 2024-09-16T09:02:51.763 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.759+0000 7f59eb06b640 1 -- 172.21.15.160:0/1631819600 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f59e41a7930 con 0x7f59e4101cb0 2024-09-16T09:02:51.763 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.759+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f59d80016b0 con 0x7f59e4101cb0 2024-09-16T09:02:51.763 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.759+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f59d8005d20 con 0x7f59e4101cb0 2024-09-16T09:02:51.763 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.759+0000 7f59eb06b640 1 -- 172.21.15.160:0/1631819600 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f59e41a7f70 con 0x7f59e4101cb0 2024-09-16T09:02:51.763 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.760+0000 7f59eb06b640 1 -- 172.21.15.160:0/1631819600 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e4102d20 con 0x7f59e4101cb0 2024-09-16T09:02:51.763 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.760+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7f59d80038a0 con 0x7f59e4101cb0 2024-09-16T09:02:51.764 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.761+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(756..756 src has 1..756) ==== 8189+0+0 (unknown 567274311 0 0) 0x7f59d8099be0 con 0x7f59e4101cb0 2024-09-16T09:02:51.765 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.765+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f59d8064b50 con 0x7f59e4101cb0 2024-09-16T09:02:51.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:51 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2206000615' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pg_num": 12} : dispatch 2024-09-16T09:02:51.945 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:51.944+0000 7f59eb06b640 1 -- 172.21.15.160:0/1631819600 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"} v 0) -- 0x7f59e4113f60 con 0x7f59e4101cb0 2024-09-16T09:02:51.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:51 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2206000615' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pg_num": 12} : dispatch 2024-09-16T09:02:51.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:51 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2206000615' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pg_num": 12} : dispatch 2024-09-16T09:02:52.556 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:52.555+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.2 v1:172.21.15.160:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"}]=0 enabled application 'rados' on pool 'b3210054-d6e7-4974-b854-ef9506616c26' v757) ==== 213+0+0 (unknown 3309024568 0 0) 0x7f59d8069a90 con 0x7f59e4101cb0 2024-09-16T09:02:52.670 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:52.669+0000 7f59eb06b640 1 -- 172.21.15.160:0/1631819600 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"} v 0) -- 0x7f59e41a8490 con 0x7f59e4101cb0 2024-09-16T09:02:52.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:52 smithi169 ceph-mon[33465]: pgmap v1198: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2206000615' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pg_num": 12}]': finished 2024-09-16T09:02:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:52 smithi169 ceph-mon[33465]: osdmap e756: 8 total, 8 up, 8 in 2024-09-16T09:02:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/2206000615' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pg_num": 12} : dispatch 2024-09-16T09:02:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:52 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1631819600' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"} : dispatch 2024-09-16T09:02:52.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:52 smithi169 ceph-mon[33465]: from='client.57443 ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"} : dispatch 2024-09-16T09:02:52.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:52 smithi160 ceph-mon[30137]: pgmap v1198: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:52.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2206000615' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pg_num": 12}]': finished 2024-09-16T09:02:52.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:52 smithi160 ceph-mon[30137]: osdmap e756: 8 total, 8 up, 8 in 2024-09-16T09:02:52.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/2206000615' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pg_num": 12} : dispatch 2024-09-16T09:02:52.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:52 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1631819600' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"} : dispatch 2024-09-16T09:02:52.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:52 smithi160 ceph-mon[30137]: from='client.57443 ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"} : dispatch 2024-09-16T09:02:52.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:52 smithi160 ceph-mon[40028]: pgmap v1198: 176 pgs: 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:02:52.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2206000615' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pg_num": 12}]': finished 2024-09-16T09:02:52.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:52 smithi160 ceph-mon[40028]: osdmap e756: 8 total, 8 up, 8 in 2024-09-16T09:02:52.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/2206000615' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pg_num": 12} : dispatch 2024-09-16T09:02:52.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:52 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1631819600' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"} : dispatch 2024-09-16T09:02:52.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:52 smithi160 ceph-mon[40028]: from='client.57443 ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"} : dispatch 2024-09-16T09:02:53.570 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.569+0000 7f59e1ffb640 1 -- 172.21.15.160:0/1631819600 <== mon.2 v1:172.21.15.160:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"}]=0 enabled application 'rados' on pool 'b3210054-d6e7-4974-b854-ef9506616c26' v758) ==== 213+0+0 (unknown 2337910290 0 0) 0x7f59d805c250 con 0x7f59e4101cb0 2024-09-16T09:02:53.570 INFO:tasks.workunit.client.0.smithi160.stderr:enabled application 'rados' on pool 'b3210054-d6e7-4974-b854-ef9506616c26' 2024-09-16T09:02:53.573 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.572+0000 7f59eb06b640 1 -- 172.21.15.160:0/1631819600 >> v1:172.21.15.160:6800/2859204856 conn(0x7f59d407bbf0 legacy=0x7f59d407e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:53.573 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.572+0000 7f59eb06b640 1 -- 172.21.15.160:0/1631819600 >> v1:172.21.15.160:6790/0 conn(0x7f59e4101cb0 legacy=0x7f59e410d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:53.573 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.572+0000 7f59eb06b640 1 -- 172.21.15.160:0/1631819600 shutdown_connections 2024-09-16T09:02:53.573 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.572+0000 7f59eb06b640 1 -- 172.21.15.160:0/1631819600 >> 172.21.15.160:0/1631819600 conn(0x7f59e40773e0 msgr2=0x7f59e4111620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:53.573 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.572+0000 7f59eb06b640 1 -- 172.21.15.160:0/1631819600 shutdown_connections 2024-09-16T09:02:53.573 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.572+0000 7f59eb06b640 1 -- 172.21.15.160:0/1631819600 wait complete. 2024-09-16T09:02:53.584 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph osd pool set-quota b3210054-d6e7-4974-b854-ef9506616c26 max_objects 10 2024-09-16T09:02:53.672 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.671+0000 7f2f394ed640 1 -- 172.21.15.160:0/570473522 >> v1:172.21.15.160:6789/0 conn(0x7f2f3410b880 legacy=0x7f2f3410dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:53.672 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.671+0000 7f2f394ed640 1 -- 172.21.15.160:0/570473522 shutdown_connections 2024-09-16T09:02:53.672 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.671+0000 7f2f394ed640 1 -- 172.21.15.160:0/570473522 >> 172.21.15.160:0/570473522 conn(0x7f2f340fd470 msgr2=0x7f2f340ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:53.673 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.671+0000 7f2f394ed640 1 -- 172.21.15.160:0/570473522 shutdown_connections 2024-09-16T09:02:53.673 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.671+0000 7f2f394ed640 1 -- 172.21.15.160:0/570473522 wait complete. 2024-09-16T09:02:53.673 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.672+0000 7f2f394ed640 1 Processor -- start 2024-09-16T09:02:53.673 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.672+0000 7f2f394ed640 1 -- start start 2024-09-16T09:02:53.675 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.672+0000 7f2f394ed640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2f341a78d0 con 0x7f2f34103a80 2024-09-16T09:02:53.675 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.672+0000 7f2f394ed640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2f341a8ad0 con 0x7f2f3410b880 2024-09-16T09:02:53.676 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.672+0000 7f2f394ed640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2f341a9cd0 con 0x7f2f3410f390 2024-09-16T09:02:53.676 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.672+0000 7f2f32ffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f2f34103a80 0x7f2f3410b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:51418/0 (socket says 172.21.15.160:51418) 2024-09-16T09:02:53.676 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.672+0000 7f2f32ffd640 1 -- 172.21.15.160:0/3423298272 learned_addr learned my addr 172.21.15.160:0/3423298272 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T09:02:53.676 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.672+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1925085195 0 0) 0x7f2f341a9cd0 con 0x7f2f3410f390 2024-09-16T09:02:53.676 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.672+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f20003520 con 0x7f2f3410f390 2024-09-16T09:02:53.676 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.672+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1173471057 0 0) 0x7f2f341a78d0 con 0x7f2f34103a80 2024-09-16T09:02:53.676 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.672+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f341a9cd0 con 0x7f2f34103a80 2024-09-16T09:02:53.676 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2398718450 0 0) 0x7f2f341a8ad0 con 0x7f2f3410b880 2024-09-16T09:02:53.676 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f341a78d0 con 0x7f2f3410b880 2024-09-16T09:02:53.676 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4022727051 0 0) 0x7f2f20003520 con 0x7f2f3410f390 2024-09-16T09:02:53.676 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2f341a8ad0 con 0x7f2f3410f390 2024-09-16T09:02:53.676 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f2f2c001730 con 0x7f2f3410f390 2024-09-16T09:02:53.676 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2817877306 0 0) 0x7f2f341a9cd0 con 0x7f2f34103a80 2024-09-16T09:02:53.677 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2f20003520 con 0x7f2f34103a80 2024-09-16T09:02:53.677 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f2f24001930 con 0x7f2f34103a80 2024-09-16T09:02:53.677 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 36656462 0 0) 0x7f2f341a78d0 con 0x7f2f3410b880 2024-09-16T09:02:53.677 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2f341a9cd0 con 0x7f2f3410b880 2024-09-16T09:02:53.677 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f2f1c001430 con 0x7f2f3410b880 2024-09-16T09:02:53.677 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4054241559 0 0) 0x7f2f341a8ad0 con 0x7f2f3410f390 2024-09-16T09:02:53.677 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 >> v1:172.21.15.169:6789/0 conn(0x7f2f3410b880 legacy=0x7f2f341a26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:53.677 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 >> v1:172.21.15.160:6789/0 conn(0x7f2f34103a80 legacy=0x7f2f3410b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:53.677 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f341aaf70 con 0x7f2f3410f390 2024-09-16T09:02:53.677 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f394ed640 1 -- 172.21.15.160:0/3423298272 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2f341a7b00 con 0x7f2f3410f390 2024-09-16T09:02:53.677 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f394ed640 1 -- 172.21.15.160:0/3423298272 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f2f341a80c0 con 0x7f2f3410f390 2024-09-16T09:02:53.677 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f2f2c001950 con 0x7f2f3410f390 2024-09-16T09:02:53.677 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.673+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f2f2c004d70 con 0x7f2f3410f390 2024-09-16T09:02:53.678 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.674+0000 7f2f394ed640 1 -- 172.21.15.160:0/3423298272 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f140050b0 con 0x7f2f3410f390 2024-09-16T09:02:53.678 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.675+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7f2f2c003c20 con 0x7f2f3410f390 2024-09-16T09:02:53.678 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.675+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(758..758 src has 1..758) ==== 8202+0+0 (unknown 550222238 0 0) 0x7f2f2c098bf0 con 0x7f2f3410f390 2024-09-16T09:02:53.680 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.680+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f2f2c063a20 con 0x7f2f3410f390 2024-09-16T09:02:53.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:53 smithi169 ceph-mon[33465]: from='client.57443 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"}]': finished 2024-09-16T09:02:53.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:53 smithi169 ceph-mon[33465]: osdmap e757: 8 total, 8 up, 8 in 2024-09-16T09:02:53.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:53 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1631819600' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"} : dispatch 2024-09-16T09:02:53.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:53 smithi169 ceph-mon[33465]: from='client.57443 ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"} : dispatch 2024-09-16T09:02:53.857 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:53.856+0000 7f2f394ed640 1 -- 172.21.15.160:0/3423298272 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"} v 0) -- 0x7f2f140053a0 con 0x7f2f3410f390 2024-09-16T09:02:53.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:53 smithi160 ceph-mon[30137]: from='client.57443 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"}]': finished 2024-09-16T09:02:53.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:53 smithi160 ceph-mon[30137]: osdmap e757: 8 total, 8 up, 8 in 2024-09-16T09:02:53.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:53 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1631819600' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"} : dispatch 2024-09-16T09:02:53.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:53 smithi160 ceph-mon[30137]: from='client.57443 ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"} : dispatch 2024-09-16T09:02:53.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:53 smithi160 ceph-mon[40028]: from='client.57443 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"}]': finished 2024-09-16T09:02:53.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:53 smithi160 ceph-mon[40028]: osdmap e757: 8 total, 8 up, 8 in 2024-09-16T09:02:53.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:53 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1631819600' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"} : dispatch 2024-09-16T09:02:53.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:53 smithi160 ceph-mon[40028]: from='client.57443 ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"} : dispatch 2024-09-16T09:02:54.575 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:54.575+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.2 v1:172.21.15.160:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool b3210054-d6e7-4974-b854-ef9506616c26 v759) ==== 223+0+0 (unknown 3928108926 0 0) 0x7f2f2c068960 con 0x7f2f3410f390 2024-09-16T09:02:54.685 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:54.684+0000 7f2f394ed640 1 -- 172.21.15.160:0/3423298272 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"} v 0) -- 0x7f2f14005c70 con 0x7f2f3410f390 2024-09-16T09:02:54.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:54 smithi169 ceph-mon[33465]: pgmap v1201: 188 pgs: 12 unknown, 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail 2024-09-16T09:02:54.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:54 smithi169 ceph-mon[33465]: from='client.57443 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"}]': finished 2024-09-16T09:02:54.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:54 smithi169 ceph-mon[33465]: osdmap e758: 8 total, 8 up, 8 in 2024-09-16T09:02:54.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:54 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3423298272' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:02:54.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:54 smithi169 ceph-mon[33465]: from='client.57449 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:02:54.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:54 smithi160 ceph-mon[30137]: pgmap v1201: 188 pgs: 12 unknown, 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail 2024-09-16T09:02:54.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:54 smithi160 ceph-mon[30137]: from='client.57443 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"}]': finished 2024-09-16T09:02:54.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:54 smithi160 ceph-mon[30137]: osdmap e758: 8 total, 8 up, 8 in 2024-09-16T09:02:54.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:54 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3423298272' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:02:54.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:54 smithi160 ceph-mon[30137]: from='client.57449 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:02:54.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:54 smithi160 ceph-mon[40028]: pgmap v1201: 188 pgs: 12 unknown, 176 active+clean; 733 KiB data, 795 MiB used, 714 GiB / 715 GiB avail 2024-09-16T09:02:54.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:54 smithi160 ceph-mon[40028]: from='client.57443 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "app": "rados"}]': finished 2024-09-16T09:02:54.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:54 smithi160 ceph-mon[40028]: osdmap e758: 8 total, 8 up, 8 in 2024-09-16T09:02:54.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:54 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3423298272' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:02:54.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:54 smithi160 ceph-mon[40028]: from='client.57449 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:02:55.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:02:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:02:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:02:55.585 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:55.584+0000 7f2f30ff9640 1 -- 172.21.15.160:0/3423298272 <== mon.2 v1:172.21.15.160:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool b3210054-d6e7-4974-b854-ef9506616c26 v760) ==== 223+0+0 (unknown 1357112765 0 0) 0x7f2f2c0046f0 con 0x7f2f3410f390 2024-09-16T09:02:55.586 INFO:tasks.workunit.client.0.smithi160.stderr:set-quota max_objects = 10 for pool b3210054-d6e7-4974-b854-ef9506616c26 2024-09-16T09:02:55.589 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:55.588+0000 7f2f394ed640 1 -- 172.21.15.160:0/3423298272 >> v1:172.21.15.160:6800/2859204856 conn(0x7f2f2007bb70 legacy=0x7f2f2007e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:55.589 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:55.588+0000 7f2f394ed640 1 -- 172.21.15.160:0/3423298272 >> v1:172.21.15.160:6790/0 conn(0x7f2f3410f390 legacy=0x7f2f341a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:55.589 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:55.588+0000 7f2f394ed640 1 -- 172.21.15.160:0/3423298272 shutdown_connections 2024-09-16T09:02:55.589 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:55.588+0000 7f2f394ed640 1 -- 172.21.15.160:0/3423298272 >> 172.21.15.160:0/3423298272 conn(0x7f2f340fd470 msgr2=0x7f2f340ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:55.589 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:55.588+0000 7f2f394ed640 1 -- 172.21.15.160:0/3423298272 shutdown_connections 2024-09-16T09:02:55.589 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:02:55.588+0000 7f2f394ed640 1 -- 172.21.15.160:0/3423298272 wait complete. 2024-09-16T09:02:55.600 INFO:tasks.workunit.client.0.smithi160.stderr:+ sleep 30 2024-09-16T09:02:55.850 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:02:55 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:02:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:55 smithi169 ceph-mon[33465]: from='client.57449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"}]': finished 2024-09-16T09:02:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:55 smithi169 ceph-mon[33465]: osdmap e759: 8 total, 8 up, 8 in 2024-09-16T09:02:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:55 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3423298272' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:02:55.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:55 smithi169 ceph-mon[33465]: from='client.57449 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:02:55.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:55 smithi160 ceph-mon[30137]: from='client.57449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"}]': finished 2024-09-16T09:02:55.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:55 smithi160 ceph-mon[30137]: osdmap e759: 8 total, 8 up, 8 in 2024-09-16T09:02:55.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:55 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3423298272' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:02:55.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:55 smithi160 ceph-mon[30137]: from='client.57449 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:02:55.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:55 smithi160 ceph-mon[40028]: from='client.57449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"}]': finished 2024-09-16T09:02:55.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:55 smithi160 ceph-mon[40028]: osdmap e759: 8 total, 8 up, 8 in 2024-09-16T09:02:55.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:55 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3423298272' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:02:55.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:55 smithi160 ceph-mon[40028]: from='client.57449 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"} : dispatch 2024-09-16T09:02:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:56 smithi169 ceph-mon[33465]: pgmap v1204: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-16T09:02:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:56 smithi169 ceph-mon[33465]: from='client.57449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"}]': finished 2024-09-16T09:02:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:56 smithi169 ceph-mon[33465]: osdmap e760: 8 total, 8 up, 8 in 2024-09-16T09:02:56.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:56 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:56 smithi160 ceph-mon[30137]: pgmap v1204: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-16T09:02:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:56 smithi160 ceph-mon[30137]: from='client.57449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"}]': finished 2024-09-16T09:02:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:56 smithi160 ceph-mon[30137]: osdmap e760: 8 total, 8 up, 8 in 2024-09-16T09:02:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:56 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:56.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:56 smithi160 ceph-mon[40028]: pgmap v1204: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-16T09:02:56.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:56 smithi160 ceph-mon[40028]: from='client.57449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "field": "max_objects", "val": "10"}]': finished 2024-09-16T09:02:56.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:56 smithi160 ceph-mon[40028]: osdmap e760: 8 total, 8 up, 8 in 2024-09-16T09:02:56.949 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:56 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:02:58.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:02:58 smithi169 ceph-mon[33465]: pgmap v1206: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.4 KiB/s wr, 1 op/s 2024-09-16T09:02:58.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:02:58 smithi160 ceph-mon[30137]: pgmap v1206: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.4 KiB/s wr, 1 op/s 2024-09-16T09:02:58.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:02:58 smithi160 ceph-mon[40028]: pgmap v1206: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.4 KiB/s wr, 1 op/s 2024-09-16T09:03:00.849 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:00 smithi169 ceph-mon[33465]: pgmap v1207: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-16T09:03:00.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:00 smithi160 ceph-mon[30137]: pgmap v1207: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-16T09:03:00.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:00 smithi160 ceph-mon[40028]: pgmap v1207: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-16T09:03:02.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:02 smithi169 ceph-mon[33465]: pgmap v1208: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.3 KiB/s wr, 1 op/s 2024-09-16T09:03:02.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:02 smithi160 ceph-mon[30137]: pgmap v1208: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.3 KiB/s wr, 1 op/s 2024-09-16T09:03:02.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:02 smithi160 ceph-mon[40028]: pgmap v1208: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.3 KiB/s wr, 1 op/s 2024-09-16T09:03:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:04 smithi169 ceph-mon[33465]: pgmap v1209: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 582 B/s rd, 0 op/s 2024-09-16T09:03:04.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:04 smithi160 ceph-mon[30137]: pgmap v1209: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 582 B/s rd, 0 op/s 2024-09-16T09:03:04.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:04.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:04 smithi160 ceph-mon[40028]: pgmap v1209: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 582 B/s rd, 0 op/s 2024-09-16T09:03:04.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:05.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:03:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:03:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:03:06.100 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:03:05 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:03:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:06 smithi169 ceph-mon[33465]: pgmap v1210: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:03:06.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:06 smithi160 ceph-mon[30137]: pgmap v1210: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:03:06.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:06.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:06 smithi160 ceph-mon[40028]: pgmap v1210: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:03:06.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:08.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:08 smithi169 ceph-mon[33465]: pgmap v1211: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 869 B/s rd, 0 op/s 2024-09-16T09:03:08.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:08 smithi160 ceph-mon[30137]: pgmap v1211: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 869 B/s rd, 0 op/s 2024-09-16T09:03:08.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:08 smithi160 ceph-mon[40028]: pgmap v1211: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 869 B/s rd, 0 op/s 2024-09-16T09:03:10.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:10 smithi160 ceph-mon[30137]: pgmap v1212: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:10.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:10 smithi160 ceph-mon[40028]: pgmap v1212: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:11.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:10 smithi169 ceph-mon[33465]: pgmap v1212: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:12.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:12 smithi160 ceph-mon[30137]: pgmap v1213: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:12.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:12 smithi160 ceph-mon[40028]: pgmap v1213: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:13.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:12 smithi169 ceph-mon[33465]: pgmap v1213: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:14.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:14 smithi160 ceph-mon[30137]: pgmap v1214: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:14.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:14 smithi160 ceph-mon[40028]: pgmap v1214: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:15.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:14 smithi169 ceph-mon[33465]: pgmap v1214: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:15.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:03:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:03:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:03:16.100 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:03:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:03:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:16 smithi160 ceph-mon[30137]: pgmap v1215: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:16.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:16.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:16 smithi160 ceph-mon[40028]: pgmap v1215: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:16.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:17.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:16 smithi169 ceph-mon[33465]: pgmap v1215: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:17.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:18.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:18 smithi160 ceph-mon[30137]: pgmap v1216: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-16T09:03:18.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:18 smithi160 ceph-mon[40028]: pgmap v1216: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-16T09:03:19.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:18 smithi169 ceph-mon[33465]: pgmap v1216: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-16T09:03:19.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:19 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:19.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:19 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:20.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:19 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:20.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:20 smithi160 ceph-mon[30137]: pgmap v1217: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:20.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:20 smithi160 ceph-mon[40028]: pgmap v1217: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:21.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:20 smithi169 ceph-mon[33465]: pgmap v1217: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:22.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:22 smithi160 ceph-mon[40028]: pgmap v1218: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-16T09:03:22.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:22 smithi160 ceph-mon[30137]: pgmap v1218: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-16T09:03:23.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:22 smithi169 ceph-mon[33465]: pgmap v1218: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-16T09:03:24.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:24 smithi160 ceph-mon[30137]: pgmap v1219: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:24.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:24 smithi160 ceph-mon[40028]: pgmap v1219: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:25.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:24 smithi169 ceph-mon[33465]: pgmap v1219: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:25.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:03:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:03:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:03:25.602 INFO:tasks.workunit.client.0.smithi160.stderr:++ seq 1 10 2024-09-16T09:03:25.602 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:03:25.602 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p b3210054-d6e7-4974-b854-ef9506616c26 put obj1 /etc/passwd 2024-09-16T09:03:25.644 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:03:25.645 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p b3210054-d6e7-4974-b854-ef9506616c26 put obj2 /etc/passwd 2024-09-16T09:03:25.681 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:03:25.681 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p b3210054-d6e7-4974-b854-ef9506616c26 put obj3 /etc/passwd 2024-09-16T09:03:25.717 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:03:25.717 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p b3210054-d6e7-4974-b854-ef9506616c26 put obj4 /etc/passwd 2024-09-16T09:03:25.751 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:03:25.752 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p b3210054-d6e7-4974-b854-ef9506616c26 put obj5 /etc/passwd 2024-09-16T09:03:25.785 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:03:25.785 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p b3210054-d6e7-4974-b854-ef9506616c26 put obj6 /etc/passwd 2024-09-16T09:03:25.819 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:03:25.819 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p b3210054-d6e7-4974-b854-ef9506616c26 put obj7 /etc/passwd 2024-09-16T09:03:25.854 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:03:25.854 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p b3210054-d6e7-4974-b854-ef9506616c26 put obj8 /etc/passwd 2024-09-16T09:03:25.888 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:03:25.888 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p b3210054-d6e7-4974-b854-ef9506616c26 put obj9 /etc/passwd 2024-09-16T09:03:25.921 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in `seq 1 10` 2024-09-16T09:03:25.921 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p b3210054-d6e7-4974-b854-ef9506616c26 put obj10 /etc/passwd 2024-09-16T09:03:25.955 INFO:tasks.workunit.client.0.smithi160.stderr:+ sleep 30 2024-09-16T09:03:26.100 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:03:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:03:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:26 smithi160 ceph-mon[30137]: pgmap v1220: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:26.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:26 smithi160 ceph-mon[40028]: pgmap v1220: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:26.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:27.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:26 smithi169 ceph-mon[33465]: pgmap v1220: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:27.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:28 smithi160 ceph-mon[30137]: pgmap v1221: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:28.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:28 smithi160 ceph-mon[40028]: pgmap v1221: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:29.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:28 smithi169 ceph-mon[33465]: pgmap v1221: 188 pgs: 188 active+clean; 759 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:30.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:30 smithi160 ceph-mon[30137]: pgmap v1222: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-16T09:03:30.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:30 smithi160 ceph-mon[40028]: pgmap v1222: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-16T09:03:31.101 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:30 smithi169 ceph-mon[33465]: pgmap v1222: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-16T09:03:32.902 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:32 smithi169 ceph-mon[33465]: pgmap v1223: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-16T09:03:32.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:32 smithi160 ceph-mon[30137]: pgmap v1223: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-16T09:03:32.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:32 smithi160 ceph-mon[40028]: pgmap v1223: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-16T09:03:33.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:33 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:03:33.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:33 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:03:33.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:33 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:03:33.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:33 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:03:33.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:33 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:03:33.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:33 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:03:33.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:33 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:03:33.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:33 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:03:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:33 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:03:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:33 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:03:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:33 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:03:34.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:33 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:03:34.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:34 smithi160 ceph-mon[40028]: pgmap v1224: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-16T09:03:34.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:34 smithi160 ceph-mon[40028]: pool 'b3210054-d6e7-4974-b854-ef9506616c26' is full (reached quota's max_objects: 10) 2024-09-16T09:03:34.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:34 smithi160 ceph-mon[40028]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-16T09:03:34.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:34 smithi160 ceph-mon[40028]: osdmap e761: 8 total, 8 up, 8 in 2024-09-16T09:03:34.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:03:34.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:34 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:34.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:34 smithi160 ceph-mon[30137]: pgmap v1224: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-16T09:03:34.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:34 smithi160 ceph-mon[30137]: pool 'b3210054-d6e7-4974-b854-ef9506616c26' is full (reached quota's max_objects: 10) 2024-09-16T09:03:34.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:34 smithi160 ceph-mon[30137]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-16T09:03:34.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:34 smithi160 ceph-mon[30137]: osdmap e761: 8 total, 8 up, 8 in 2024-09-16T09:03:34.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:03:34.949 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:34 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:35.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:34 smithi169 ceph-mon[33465]: pgmap v1224: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-16T09:03:35.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:34 smithi169 ceph-mon[33465]: pool 'b3210054-d6e7-4974-b854-ef9506616c26' is full (reached quota's max_objects: 10) 2024-09-16T09:03:35.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:34 smithi169 ceph-mon[33465]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-16T09:03:35.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:34 smithi169 ceph-mon[33465]: osdmap e761: 8 total, 8 up, 8 in 2024-09-16T09:03:35.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' 2024-09-16T09:03:35.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:34 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:35.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:03:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:03:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:03:36.100 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:03:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:03:36.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:36 smithi160 ceph-mon[30137]: pgmap v1226: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-16T09:03:36.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:36 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:36.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:36 smithi160 ceph-mon[40028]: pgmap v1226: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-16T09:03:36.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:36 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:37.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:36 smithi169 ceph-mon[33465]: pgmap v1226: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-16T09:03:37.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:36 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:38.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:38 smithi160 ceph-mon[30137]: pgmap v1227: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-16T09:03:38.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:38 smithi160 ceph-mon[40028]: pgmap v1227: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-16T09:03:39.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:38 smithi169 ceph-mon[33465]: pgmap v1227: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-16T09:03:40.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:40 smithi160 ceph-mon[30137]: pgmap v1228: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:03:40.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:40 smithi160 ceph-mon[40028]: pgmap v1228: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:03:41.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:40 smithi169 ceph-mon[33465]: pgmap v1228: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:03:42.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:42 smithi160 ceph-mon[30137]: pgmap v1229: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:03:42.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:42 smithi160 ceph-mon[40028]: pgmap v1229: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:03:43.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:42 smithi169 ceph-mon[33465]: pgmap v1229: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:03:44.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:44 smithi160 ceph-mon[30137]: pgmap v1230: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:03:44.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:44 smithi160 ceph-mon[40028]: pgmap v1230: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:03:45.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:44 smithi169 ceph-mon[33465]: pgmap v1230: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:03:45.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:03:44 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:03:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:03:46.100 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:03:45 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:03:46.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:46 smithi160 ceph-mon[30137]: pgmap v1231: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 927 B/s rd, 0 op/s 2024-09-16T09:03:46.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:46 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:46.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:46 smithi160 ceph-mon[40028]: pgmap v1231: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 927 B/s rd, 0 op/s 2024-09-16T09:03:46.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:46 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:47.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:46 smithi169 ceph-mon[33465]: pgmap v1231: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 927 B/s rd, 0 op/s 2024-09-16T09:03:47.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:46 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:48.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:48 smithi160 ceph-mon[30137]: pgmap v1232: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:48.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:48 smithi160 ceph-mon[40028]: pgmap v1232: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:49.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:48 smithi169 ceph-mon[33465]: pgmap v1232: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:49.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:49 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:49.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:49 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:50.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:49 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:50.950 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:50 smithi160 ceph-mon[30137]: pgmap v1233: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:50.950 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:50 smithi160 ceph-mon[40028]: pgmap v1233: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:51.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:50 smithi169 ceph-mon[33465]: pgmap v1233: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:52.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:52 smithi160 ceph-mon[30137]: pgmap v1234: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:52.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:52 smithi160 ceph-mon[40028]: pgmap v1234: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:53.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:52 smithi169 ceph-mon[33465]: pgmap v1234: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:54.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:54 smithi160 ceph-mon[30137]: pgmap v1235: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:54.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:54 smithi160 ceph-mon[40028]: pgmap v1235: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:55.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:54 smithi169 ceph-mon[33465]: pgmap v1235: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:03:55.447 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:03:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:03:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:03:55.956 INFO:tasks.workunit.client.0.smithi160.stderr:+ rados -p 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da put threemore /etc/passwd 2024-09-16T09:03:55.996 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph osd pool set-quota 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da max_bytes 0 2024-09-16T09:03:56.084 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.083+0000 7f6e6e110640 1 -- 172.21.15.160:0/4253322869 >> v1:172.21.15.160:6790/0 conn(0x7f6e6810b960 legacy=0x7f6e6810dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:56.084 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.083+0000 7f6e6e110640 1 -- 172.21.15.160:0/4253322869 shutdown_connections 2024-09-16T09:03:56.084 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.083+0000 7f6e6e110640 1 -- 172.21.15.160:0/4253322869 >> 172.21.15.160:0/4253322869 conn(0x7f6e680fd4b0 msgr2=0x7f6e680ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:56.084 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.083+0000 7f6e6e110640 1 -- 172.21.15.160:0/4253322869 shutdown_connections 2024-09-16T09:03:56.084 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.083+0000 7f6e6e110640 1 -- 172.21.15.160:0/4253322869 wait complete. 2024-09-16T09:03:56.084 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.083+0000 7f6e6e110640 1 Processor -- start 2024-09-16T09:03:56.087 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e6e110640 1 -- start start 2024-09-16T09:03:56.087 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e6e110640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6e681a78d0 con 0x7f6e68103980 2024-09-16T09:03:56.087 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e6e110640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6e681a8ad0 con 0x7f6e6810b960 2024-09-16T09:03:56.087 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e6e110640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6e681a9cd0 con 0x7f6e6810f470 2024-09-16T09:03:56.087 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e677fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f6e68103980 0x7f6e6810b100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:48604/0 (socket says 172.21.15.160:48604) 2024-09-16T09:03:56.087 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e677fe640 1 -- 172.21.15.160:0/1734794881 learned_addr learned my addr 172.21.15.160:0/1734794881 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T09:03:56.087 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 109455138 0 0) 0x7f6e681a78d0 con 0x7f6e68103980 2024-09-16T09:03:56.087 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e50003520 con 0x7f6e68103980 2024-09-16T09:03:56.087 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2836642663 0 0) 0x7f6e681a9cd0 con 0x7f6e6810f470 2024-09-16T09:03:56.088 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e681a78d0 con 0x7f6e6810f470 2024-09-16T09:03:56.088 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2961046178 0 0) 0x7f6e681a8ad0 con 0x7f6e6810b960 2024-09-16T09:03:56.088 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e681a9cd0 con 0x7f6e6810b960 2024-09-16T09:03:56.088 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1479753954 0 0) 0x7f6e50003520 con 0x7f6e68103980 2024-09-16T09:03:56.088 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6e681a8ad0 con 0x7f6e68103980 2024-09-16T09:03:56.088 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f6e58001310 con 0x7f6e68103980 2024-09-16T09:03:56.088 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4154117986 0 0) 0x7f6e681a78d0 con 0x7f6e6810f470 2024-09-16T09:03:56.088 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6e50003520 con 0x7f6e6810f470 2024-09-16T09:03:56.088 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f6e60001b70 con 0x7f6e6810f470 2024-09-16T09:03:56.088 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2276111152 0 0) 0x7f6e681a9cd0 con 0x7f6e6810b960 2024-09-16T09:03:56.088 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6e681a78d0 con 0x7f6e6810b960 2024-09-16T09:03:56.088 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.084+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f6e5c001430 con 0x7f6e6810b960 2024-09-16T09:03:56.089 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.085+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2113797791 0 0) 0x7f6e681a8ad0 con 0x7f6e68103980 2024-09-16T09:03:56.089 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.085+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 >> v1:172.21.15.160:6790/0 conn(0x7f6e6810f470 legacy=0x7f6e681a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:56.089 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.085+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 >> v1:172.21.15.169:6789/0 conn(0x7f6e6810b960 legacy=0x7f6e681a2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:56.089 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.085+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e681aaf70 con 0x7f6e68103980 2024-09-16T09:03:56.089 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.085+0000 7f6e6e110640 1 -- 172.21.15.160:0/1734794881 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6e681a9f00 con 0x7f6e68103980 2024-09-16T09:03:56.089 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.085+0000 7f6e6e110640 1 -- 172.21.15.160:0/1734794881 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f6e681aa540 con 0x7f6e68103980 2024-09-16T09:03:56.089 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.085+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f6e58003a60 con 0x7f6e68103980 2024-09-16T09:03:56.089 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.085+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f6e58005c10 con 0x7f6e68103980 2024-09-16T09:03:56.089 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.086+0000 7f6e6e110640 1 -- 172.21.15.160:0/1734794881 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e540050b0 con 0x7f6e68103980 2024-09-16T09:03:56.089 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.086+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7f6e580035f0 con 0x7f6e68103980 2024-09-16T09:03:56.090 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.087+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(761..761 src has 1..761) ==== 8202+0+0 (unknown 3300147437 0 0) 0x7f6e58099910 con 0x7f6e68103980 2024-09-16T09:03:56.090 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.087+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=762}) -- 0x7f6e681a8ad0 con 0x7f6e68103980 2024-09-16T09:03:56.092 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.091+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f6e58064740 con 0x7f6e68103980 2024-09-16T09:03:56.099 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:03:55 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:03:56.270 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.269+0000 7f6e6e110640 1 -- 172.21.15.160:0/1734794881 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} v 0) -- 0x7f6e540053a0 con 0x7f6e68103980 2024-09-16T09:03:56.685 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.685+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da v762) ==== 217+0+0 (unknown 4071100641 0 0) 0x7f6e58069680 con 0x7f6e68103980 2024-09-16T09:03:56.686 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.685+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.0 v1:172.21.15.160:6789/0 11 ==== osd_map(762..762 src has 1..762) ==== 628+0+0 (unknown 816077731 0 0) 0x7f6e58097540 con 0x7f6e68103980 2024-09-16T09:03:56.686 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.685+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=763}) -- 0x7f6e681a9cd0 con 0x7f6e68103980 2024-09-16T09:03:56.796 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:56.795+0000 7f6e6e110640 1 -- 172.21.15.160:0/1734794881 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} v 0) -- 0x7f6e54005c70 con 0x7f6e68103980 2024-09-16T09:03:56.947 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:56 smithi160 ceph-mon[40028]: pgmap v1236: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:56.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:56 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:56.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:56 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1734794881' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:03:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:56 smithi160 ceph-mon[30137]: pgmap v1236: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:56 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:56.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:56 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1734794881' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:03:57.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:56 smithi169 ceph-mon[33465]: pgmap v1236: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:03:57.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:56 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:03:57.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:56 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1734794881' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:03:57.690 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.690+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.0 v1:172.21.15.160:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da v763) ==== 217+0+0 (unknown 603135279 0 0) 0x7f6e580691f0 con 0x7f6e68103980 2024-09-16T09:03:57.691 INFO:tasks.workunit.client.0.smithi160.stderr:set-quota max_bytes = 0 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da 2024-09-16T09:03:57.691 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.690+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 <== mon.0 v1:172.21.15.160:6789/0 13 ==== osd_map(763..763 src has 1..763) ==== 628+0+0 (unknown 450428467 0 0) 0x7f6e5805e930 con 0x7f6e68103980 2024-09-16T09:03:57.691 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.690+0000 7f6e64ff9640 1 -- 172.21.15.160:0/1734794881 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=764}) -- 0x7f6e50089740 con 0x7f6e68103980 2024-09-16T09:03:57.694 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.693+0000 7f6e6e110640 1 -- 172.21.15.160:0/1734794881 >> v1:172.21.15.160:6800/2859204856 conn(0x7f6e5007bcd0 legacy=0x7f6e5007e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:57.694 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.693+0000 7f6e6e110640 1 -- 172.21.15.160:0/1734794881 >> v1:172.21.15.160:6789/0 conn(0x7f6e68103980 legacy=0x7f6e6810b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:57.695 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.694+0000 7f6e6e110640 1 -- 172.21.15.160:0/1734794881 shutdown_connections 2024-09-16T09:03:57.695 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.694+0000 7f6e6e110640 1 -- 172.21.15.160:0/1734794881 >> 172.21.15.160:0/1734794881 conn(0x7f6e680fd4b0 msgr2=0x7f6e680fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:57.696 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.695+0000 7f6e6e110640 1 -- 172.21.15.160:0/1734794881 shutdown_connections 2024-09-16T09:03:57.696 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.695+0000 7f6e6e110640 1 -- 172.21.15.160:0/1734794881 wait complete. 2024-09-16T09:03:57.710 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph osd pool set-quota 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da max_objects 0 2024-09-16T09:03:57.800 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.795+0000 7f714a247640 1 -- 172.21.15.160:0/798934376 >> v1:172.21.15.160:6789/0 conn(0x7f7144101aa0 legacy=0x7f7144101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:57.800 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.795+0000 7f714a247640 1 -- 172.21.15.160:0/798934376 shutdown_connections 2024-09-16T09:03:57.800 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.795+0000 7f714a247640 1 -- 172.21.15.160:0/798934376 >> 172.21.15.160:0/798934376 conn(0x7f71440fd4a0 msgr2=0x7f71440ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:57.800 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.795+0000 7f714a247640 1 -- 172.21.15.160:0/798934376 shutdown_connections 2024-09-16T09:03:57.800 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.796+0000 7f714a247640 1 -- 172.21.15.160:0/798934376 wait complete. 2024-09-16T09:03:57.801 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.796+0000 7f714a247640 1 Processor -- start 2024-09-16T09:03:57.801 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.796+0000 7f714a247640 1 -- start start 2024-09-16T09:03:57.801 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.796+0000 7f714a247640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f71441a7870 con 0x7f714410e1d0 2024-09-16T09:03:57.801 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.796+0000 7f714a247640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f71441a8a70 con 0x7f7144101aa0 2024-09-16T09:03:57.801 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.796+0000 7f714a247640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f71441a9c70 con 0x7f714410a600 2024-09-16T09:03:57.801 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7143fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f714410e1d0 0x7f71441a5f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:39346/0 (socket says 172.21.15.160:39346) 2024-09-16T09:03:57.801 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7143fff640 1 -- 172.21.15.160:0/3795244929 learned_addr learned my addr 172.21.15.160:0/3795244929 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T09:03:57.801 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7142ffd640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7f714410a600 0x7f71441a2610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:42994/0 (socket says 172.21.15.160:42994) 2024-09-16T09:03:57.801 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3809658687 0 0) 0x7f71441a9c70 con 0x7f714410a600 2024-09-16T09:03:57.801 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7130003160 con 0x7f714410a600 2024-09-16T09:03:57.801 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2694505940 0 0) 0x7f71441a7870 con 0x7f714410e1d0 2024-09-16T09:03:57.801 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71441a9c70 con 0x7f714410e1d0 2024-09-16T09:03:57.802 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1088817472 0 0) 0x7f71441a8a70 con 0x7f7144101aa0 2024-09-16T09:03:57.802 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71441a7870 con 0x7f7144101aa0 2024-09-16T09:03:57.802 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 735387831 0 0) 0x7f7130003160 con 0x7f714410a600 2024-09-16T09:03:57.802 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f71441a8a70 con 0x7f714410a600 2024-09-16T09:03:57.802 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f712c001410 con 0x7f714410a600 2024-09-16T09:03:57.802 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2595128998 0 0) 0x7f71441a9c70 con 0x7f714410e1d0 2024-09-16T09:03:57.802 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7130003160 con 0x7f714410e1d0 2024-09-16T09:03:57.802 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f713c001c30 con 0x7f714410e1d0 2024-09-16T09:03:57.802 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4056527259 0 0) 0x7f71441a7870 con 0x7f7144101aa0 2024-09-16T09:03:57.802 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 --> v1:172.21.15.169:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f71441a9c70 con 0x7f7144101aa0 2024-09-16T09:03:57.802 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f71340015b0 con 0x7f7144101aa0 2024-09-16T09:03:57.802 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.2 v1:172.21.15.160:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3705119551 0 0) 0x7f71441a8a70 con 0x7f714410a600 2024-09-16T09:03:57.802 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 >> v1:172.21.15.169:6789/0 conn(0x7f7144101aa0 legacy=0x7f714406b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:57.802 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 >> v1:172.21.15.160:6789/0 conn(0x7f714410e1d0 legacy=0x7f71441a5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:57.803 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 --> v1:172.21.15.160:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71441aaf10 con 0x7f714410a600 2024-09-16T09:03:57.803 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f714a247640 1 -- 172.21.15.160:0/3795244929 --> v1:172.21.15.160:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f71441a7aa0 con 0x7f714410a600 2024-09-16T09:03:57.803 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f714a247640 1 -- 172.21.15.160:0/3795244929 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f71441a8060 con 0x7f714410a600 2024-09-16T09:03:57.803 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.2 v1:172.21.15.160:6790/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f712c003a60 con 0x7f714410a600 2024-09-16T09:03:57.803 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.797+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.2 v1:172.21.15.160:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f712c005b50 con 0x7f714410a600 2024-09-16T09:03:57.803 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.798+0000 7f714a247640 1 -- 172.21.15.160:0/3795244929 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71240050b0 con 0x7f714410a600 2024-09-16T09:03:57.803 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.799+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.2 v1:172.21.15.160:6790/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7f712c0035f0 con 0x7f714410a600 2024-09-16T09:03:57.803 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.800+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.2 v1:172.21.15.160:6790/0 8 ==== osd_map(763..763 src has 1..763) ==== 8202+0+0 (unknown 3814371175 0 0) 0x7f712c099930 con 0x7f714410a600 2024-09-16T09:03:57.803 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.800+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=764}) -- 0x7f71441a8a70 con 0x7f714410a600 2024-09-16T09:03:57.804 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.804+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.2 v1:172.21.15.160:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f712c064760 con 0x7f714410a600 2024-09-16T09:03:57.947 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1734794881' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"}]': finished 2024-09-16T09:03:57.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:57 smithi160 ceph-mon[30137]: osdmap e762: 8 total, 8 up, 8 in 2024-09-16T09:03:57.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:57 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1734794881' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:03:57.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:57 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1734794881' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"}]': finished 2024-09-16T09:03:57.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:57 smithi160 ceph-mon[40028]: osdmap e762: 8 total, 8 up, 8 in 2024-09-16T09:03:57.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:57 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1734794881' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:03:57.984 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:57.984+0000 7f714a247640 1 -- 172.21.15.160:0/3795244929 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} v 0) -- 0x7f71240053a0 con 0x7f714410a600 2024-09-16T09:03:58.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:57 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1734794881' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"}]': finished 2024-09-16T09:03:58.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:57 smithi169 ceph-mon[33465]: osdmap e762: 8 total, 8 up, 8 in 2024-09-16T09:03:58.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:57 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1734794881' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"} : dispatch 2024-09-16T09:03:58.698 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:58.697+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.2 v1:172.21.15.160:6790/0 10 ==== osd_map(764..764 src has 1..764) ==== 628+0+0 (unknown 1039014637 0 0) 0x7f712c097560 con 0x7f714410a600 2024-09-16T09:03:58.698 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:58.697+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=765}) -- 0x7f7130003160 con 0x7f714410a600 2024-09-16T09:03:58.702 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:58.702+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.2 v1:172.21.15.160:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da v764) ==== 221+0+0 (unknown 1784508741 0 0) 0x7f712c0696a0 con 0x7f714410a600 2024-09-16T09:03:58.814 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:58.814+0000 7f714a247640 1 -- 172.21.15.160:0/3795244929 --> v1:172.21.15.160:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} v 0) -- 0x7f7124005c70 con 0x7f714410a600 2024-09-16T09:03:59.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:58 smithi169 ceph-mon[33465]: pgmap v1238: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:03:59.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/1734794881' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"}]': finished 2024-09-16T09:03:59.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:58 smithi169 ceph-mon[33465]: osdmap e763: 8 total, 8 up, 8 in 2024-09-16T09:03:59.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:58 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3795244929' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:03:59.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:58 smithi169 ceph-mon[33465]: from='client.57560 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:03:59.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:58 smithi160 ceph-mon[30137]: pgmap v1238: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:03:59.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/1734794881' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"}]': finished 2024-09-16T09:03:59.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:58 smithi160 ceph-mon[30137]: osdmap e763: 8 total, 8 up, 8 in 2024-09-16T09:03:59.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:58 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3795244929' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:03:59.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:58 smithi160 ceph-mon[30137]: from='client.57560 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:03:59.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:58 smithi160 ceph-mon[40028]: pgmap v1238: 188 pgs: 188 active+clean; 884 KiB data, 800 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:03:59.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/1734794881' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_bytes", "val": "0"}]': finished 2024-09-16T09:03:59.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:58 smithi160 ceph-mon[40028]: osdmap e763: 8 total, 8 up, 8 in 2024-09-16T09:03:59.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:58 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3795244929' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:03:59.199 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:58 smithi160 ceph-mon[40028]: from='client.57560 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:03:59.719 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:59.719+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.2 v1:172.21.15.160:6790/0 12 ==== osd_map(765..765 src has 1..765) ==== 628+0+0 (unknown 1589065864 0 0) 0x7f712c05f3f0 con 0x7f714410a600 2024-09-16T09:03:59.720 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:59.719+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 --> v1:172.21.15.160:6790/0 -- mon_subscribe({osdmap=766}) -- 0x7f71441a7870 con 0x7f714410a600 2024-09-16T09:03:59.725 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:59.724+0000 7f7140ff9640 1 -- 172.21.15.160:0/3795244929 <== mon.2 v1:172.21.15.160:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da v765) ==== 221+0+0 (unknown 3926761636 0 0) 0x7f712c069210 con 0x7f714410a600 2024-09-16T09:03:59.725 INFO:tasks.workunit.client.0.smithi160.stderr:set-quota max_objects = 0 for pool 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da 2024-09-16T09:03:59.728 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:59.728+0000 7f714a247640 1 -- 172.21.15.160:0/3795244929 >> v1:172.21.15.160:6800/2859204856 conn(0x7f713007bad0 legacy=0x7f713007df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:59.728 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:59.728+0000 7f714a247640 1 -- 172.21.15.160:0/3795244929 >> v1:172.21.15.160:6790/0 conn(0x7f714410a600 legacy=0x7f71441a2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:59.729 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:59.728+0000 7f714a247640 1 -- 172.21.15.160:0/3795244929 shutdown_connections 2024-09-16T09:03:59.729 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:59.728+0000 7f714a247640 1 -- 172.21.15.160:0/3795244929 >> 172.21.15.160:0/3795244929 conn(0x7f71440fd4a0 msgr2=0x7f71441052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:59.729 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:59.728+0000 7f714a247640 1 -- 172.21.15.160:0/3795244929 shutdown_connections 2024-09-16T09:03:59.729 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:03:59.728+0000 7f714a247640 1 -- 172.21.15.160:0/3795244929 wait complete. 2024-09-16T09:03:59.739 INFO:tasks.workunit.client.0.smithi160.stderr:+ sleep 30 2024-09-16T09:04:00.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:59 smithi169 ceph-mon[33465]: from='client.57560 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"}]': finished 2024-09-16T09:04:00.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:59 smithi169 ceph-mon[33465]: osdmap e764: 8 total, 8 up, 8 in 2024-09-16T09:04:00.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:59 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3795244929' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:04:00.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:03:59 smithi169 ceph-mon[33465]: from='client.57560 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:04:00.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:59 smithi160 ceph-mon[30137]: from='client.57560 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"}]': finished 2024-09-16T09:04:00.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:59 smithi160 ceph-mon[30137]: osdmap e764: 8 total, 8 up, 8 in 2024-09-16T09:04:00.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:59 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3795244929' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:04:00.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:03:59 smithi160 ceph-mon[30137]: from='client.57560 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:04:00.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:59 smithi160 ceph-mon[40028]: from='client.57560 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"}]': finished 2024-09-16T09:04:00.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:59 smithi160 ceph-mon[40028]: osdmap e764: 8 total, 8 up, 8 in 2024-09-16T09:04:00.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:59 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3795244929' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:04:00.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:03:59 smithi160 ceph-mon[40028]: from='client.57560 ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"} : dispatch 2024-09-16T09:04:01.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:00 smithi169 ceph-mon[33465]: pgmap v1241: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-16T09:04:01.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:00 smithi169 ceph-mon[33465]: from='client.57560 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"}]': finished 2024-09-16T09:04:01.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:00 smithi169 ceph-mon[33465]: osdmap e765: 8 total, 8 up, 8 in 2024-09-16T09:04:01.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:00 smithi160 ceph-mon[30137]: pgmap v1241: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-16T09:04:01.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:00 smithi160 ceph-mon[30137]: from='client.57560 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"}]': finished 2024-09-16T09:04:01.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:00 smithi160 ceph-mon[30137]: osdmap e765: 8 total, 8 up, 8 in 2024-09-16T09:04:01.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:00 smithi160 ceph-mon[40028]: pgmap v1241: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-16T09:04:01.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:00 smithi160 ceph-mon[40028]: from='client.57560 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "field": "max_objects", "val": "0"}]': finished 2024-09-16T09:04:01.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:00 smithi160 ceph-mon[40028]: osdmap e765: 8 total, 8 up, 8 in 2024-09-16T09:04:03.000 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:02 smithi160 ceph-mon[30137]: pgmap v1243: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.8 KiB/s wr, 1 op/s 2024-09-16T09:04:03.000 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:02 smithi160 ceph-mon[40028]: pgmap v1243: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.8 KiB/s wr, 1 op/s 2024-09-16T09:04:03.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:02 smithi169 ceph-mon[33465]: pgmap v1243: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.8 KiB/s wr, 1 op/s 2024-09-16T09:04:04.989 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:04 smithi160 ceph-mon[30137]: pgmap v1244: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-16T09:04:04.990 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:04 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:04:04.990 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:04 smithi160 ceph-mon[40028]: pgmap v1244: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-16T09:04:04.990 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:04 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:04:05.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:04 smithi169 ceph-mon[33465]: pgmap v1244: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-16T09:04:05.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:04 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:04:05.242 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:04 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:04:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:04:06.100 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:04:05 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:04:07.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:06 smithi169 ceph-mon[33465]: pgmap v1245: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-16T09:04:07.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:06 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:04:07.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:06 smithi160 ceph-mon[30137]: pgmap v1245: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-16T09:04:07.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:06 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:04:07.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:06 smithi160 ceph-mon[40028]: pgmap v1245: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-16T09:04:07.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:06 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:04:09.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:08 smithi169 ceph-mon[33465]: pgmap v1246: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-09-16T09:04:09.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:08 smithi160 ceph-mon[30137]: pgmap v1246: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-09-16T09:04:09.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:08 smithi160 ceph-mon[40028]: pgmap v1246: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-09-16T09:04:11.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:10 smithi169 ceph-mon[33465]: pgmap v1247: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:04:11.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:10 smithi160 ceph-mon[30137]: pgmap v1247: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:04:11.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:10 smithi160 ceph-mon[40028]: pgmap v1247: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:04:13.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:12 smithi169 ceph-mon[33465]: pgmap v1248: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 877 B/s rd, 0 op/s 2024-09-16T09:04:13.197 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:12 smithi160 ceph-mon[30137]: pgmap v1248: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 877 B/s rd, 0 op/s 2024-09-16T09:04:13.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:12 smithi160 ceph-mon[40028]: pgmap v1248: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 877 B/s rd, 0 op/s 2024-09-16T09:04:14.989 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:14 smithi160 ceph-mon[30137]: pgmap v1249: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:04:14.990 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:14 smithi160 ceph-mon[40028]: pgmap v1249: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:04:15.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:14 smithi169 ceph-mon[33465]: pgmap v1249: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:04:15.313 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:14 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:04:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:04:16.100 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:04:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:04:17.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:16 smithi169 ceph-mon[33465]: pgmap v1250: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:04:17.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:16 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:04:17.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:16 smithi160 ceph-mon[30137]: pgmap v1250: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:04:17.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:16 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:04:17.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:16 smithi160 ceph-mon[40028]: pgmap v1250: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:04:17.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:16 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:04:18.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:17 smithi169 ceph-mon[33465]: pgmap v1251: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:04:18.198 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:17 smithi160 ceph-mon[30137]: pgmap v1251: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:04:18.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:17 smithi160 ceph-mon[40028]: pgmap v1251: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:04:20.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:20 smithi160 ceph-mon[30137]: pgmap v1252: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:04:20.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:20 smithi160 ceph-mon[30137]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:04:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:20 smithi160 ceph-mon[40028]: pgmap v1252: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:04:20.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:20 smithi160 ceph-mon[40028]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:04:20.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:20 smithi169 ceph-mon[33465]: pgmap v1252: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:04:20.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:20 smithi169 ceph-mon[33465]: from='mgr.34526 v1:172.21.15.160:0/176448304' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:04:22.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:22 smithi160 ceph-mon[30137]: pgmap v1253: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:04:22.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:22 smithi160 ceph-mon[40028]: pgmap v1253: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:04:22.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:22 smithi169 ceph-mon[33465]: pgmap v1253: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:04:24.698 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:24 smithi160 ceph-mon[30137]: pgmap v1254: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:04:24.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:24 smithi160 ceph-mon[40028]: pgmap v1254: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:04:24.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:24 smithi169 ceph-mon[33465]: pgmap v1254: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:04:25.448 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:24 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:04:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:04:26.100 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:04:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:04:26.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:26 smithi169 ceph-mon[33465]: pgmap v1255: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:04:26.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:26 smithi169 ceph-mon[33465]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:04:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:26 smithi160 ceph-mon[30137]: pgmap v1255: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:04:26.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:26 smithi160 ceph-mon[30137]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:04:26.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:26 smithi160 ceph-mon[40028]: pgmap v1255: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:04:26.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:26 smithi160 ceph-mon[40028]: from='client.34481 v1:172.21.15.169:0/2596507723' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:04:28.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:28 smithi169 ceph-mon[33465]: pgmap v1256: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:04:28.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:28 smithi160 ceph-mon[30137]: pgmap v1256: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:04:28.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:28 smithi160 ceph-mon[40028]: pgmap v1256: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:04:29.741 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph osd pool delete 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da 0356ba9f-a8cb-465e-96c7-f3ebcc55b3da --yes-i-really-really-mean-it 2024-09-16T09:04:29.833 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.832+0000 7f330cb0e640 1 -- 172.21.15.160:0/1282354354 >> v1:172.21.15.160:6789/0 conn(0x7f330810a680 legacy=0x7f330810ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:29.834 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.832+0000 7f330cb0e640 1 -- 172.21.15.160:0/1282354354 shutdown_connections 2024-09-16T09:04:29.834 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.832+0000 7f330cb0e640 1 -- 172.21.15.160:0/1282354354 >> 172.21.15.160:0/1282354354 conn(0x7f3308079180 msgr2=0x7f3308079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:29.834 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.832+0000 7f330cb0e640 1 -- 172.21.15.160:0/1282354354 shutdown_connections 2024-09-16T09:04:29.834 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.832+0000 7f330cb0e640 1 -- 172.21.15.160:0/1282354354 wait complete. 2024-09-16T09:04:29.836 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.832+0000 7f330cb0e640 1 Processor -- start 2024-09-16T09:04:29.836 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.832+0000 7f330cb0e640 1 -- start start 2024-09-16T09:04:29.836 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f330cb0e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f33081a6780 con 0x7f33081065d0 2024-09-16T09:04:29.837 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f330cb0e640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f33081a7980 con 0x7f330810a680 2024-09-16T09:04:29.837 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f330cb0e640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f33081a8b80 con 0x7f330810e1f0 2024-09-16T09:04:29.837 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f330659c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f33081065d0 0x7f3308105a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:34314/0 (socket says 172.21.15.160:34314) 2024-09-16T09:04:29.837 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f330659c640 1 -- 172.21.15.160:0/3377000355 learned_addr learned my addr 172.21.15.160:0/3377000355 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T09:04:29.837 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f3306d9d640 1 --1- 172.21.15.160:0/3377000355 >> v1:172.21.15.160:6790/0 conn(0x7f330810e1f0 0x7f33081a4e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:43658/0 (socket says 172.21.15.160:43658) 2024-09-16T09:04:29.837 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3051552631 0 0) 0x7f33081a6780 con 0x7f33081065d0 2024-09-16T09:04:29.837 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32f4003160 con 0x7f33081065d0 2024-09-16T09:04:29.837 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1582048160 0 0) 0x7f33081a8b80 con 0x7f330810e1f0 2024-09-16T09:04:29.837 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33081a6780 con 0x7f330810e1f0 2024-09-16T09:04:29.837 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 152339463 0 0) 0x7f33081a7980 con 0x7f330810a680 2024-09-16T09:04:29.837 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33081a8b80 con 0x7f330810a680 2024-09-16T09:04:29.838 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1939072522 0 0) 0x7f32f4003160 con 0x7f33081065d0 2024-09-16T09:04:29.838 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f33081a7980 con 0x7f33081065d0 2024-09-16T09:04:29.838 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f32f8001a10 con 0x7f33081065d0 2024-09-16T09:04:29.838 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 61754397 0 0) 0x7f33081a6780 con 0x7f330810e1f0 2024-09-16T09:04:29.838 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f32f4003160 con 0x7f330810e1f0 2024-09-16T09:04:29.838 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f33000015f0 con 0x7f330810e1f0 2024-09-16T09:04:29.838 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3225815121 0 0) 0x7f33081a7980 con 0x7f33081065d0 2024-09-16T09:04:29.838 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 >> v1:172.21.15.160:6790/0 conn(0x7f330810e1f0 legacy=0x7f33081a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:29.838 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.833+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 >> v1:172.21.15.169:6789/0 conn(0x7f330810a680 legacy=0x7f33081a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:29.838 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.834+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33081a9e20 con 0x7f33081065d0 2024-09-16T09:04:29.838 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.834+0000 7f330cb0e640 1 -- 172.21.15.160:0/3377000355 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f33081a8db0 con 0x7f33081065d0 2024-09-16T09:04:29.838 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.834+0000 7f330cb0e640 1 -- 172.21.15.160:0/3377000355 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f33081a9370 con 0x7f33081065d0 2024-09-16T09:04:29.838 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.834+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7f32f8001cb0 con 0x7f33081065d0 2024-09-16T09:04:29.839 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.834+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7f32f8005cc0 con 0x7f33081065d0 2024-09-16T09:04:29.839 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.835+0000 7f330cb0e640 1 -- 172.21.15.160:0/3377000355 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e80050b0 con 0x7f33081065d0 2024-09-16T09:04:29.839 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.835+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7f32f80038c0 con 0x7f33081065d0 2024-09-16T09:04:29.839 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.836+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(765..765 src has 1..765) ==== 8202+0+0 (unknown 2338955257 0 0) 0x7f32f8099a10 con 0x7f33081065d0 2024-09-16T09:04:29.840 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.836+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=766}) -- 0x7f33081a7980 con 0x7f33081065d0 2024-09-16T09:04:29.841 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:29.840+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7f32f8064840 con 0x7f33081065d0 2024-09-16T09:04:30.020 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.019+0000 7f330cb0e640 1 -- 172.21.15.160:0/3377000355 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pool2": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "yes_i_really_really_mean_it": true} v 0) -- 0x7f32e80053a0 con 0x7f33081065d0 2024-09-16T09:04:30.460 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.459+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pool2": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "yes_i_really_really_mean_it": true}]=0 pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' removed v766) ==== 248+0+0 (unknown 2423379902 0 0) 0x7f32f8069780 con 0x7f33081065d0 2024-09-16T09:04:30.463 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.462+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 <== mon.0 v1:172.21.15.160:6789/0 11 ==== osd_map(766..766 src has 1..766) ==== 296+0+0 (unknown 2779462617 0 0) 0x7f32f8097640 con 0x7f33081065d0 2024-09-16T09:04:30.463 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.462+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=767}) -- 0x7f33081a8b80 con 0x7f33081065d0 2024-09-16T09:04:30.573 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.571+0000 7f330cb0e640 1 -- 172.21.15.160:0/3377000355 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pool2": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "yes_i_really_really_mean_it": true} v 0) -- 0x7f32e8005e90 con 0x7f33081065d0 2024-09-16T09:04:30.573 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.571+0000 7f32ff7fe640 1 -- 172.21.15.160:0/3377000355 <== mon.0 v1:172.21.15.160:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pool2": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "yes_i_really_really_mean_it": true}]=0 pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' does not exist v766) ==== 255+0+0 (unknown 1443116904 0 0) 0x7f32f80057c0 con 0x7f33081065d0 2024-09-16T09:04:30.573 INFO:tasks.workunit.client.0.smithi160.stderr:pool '0356ba9f-a8cb-465e-96c7-f3ebcc55b3da' does not exist 2024-09-16T09:04:30.575 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.574+0000 7f330cb0e640 1 -- 172.21.15.160:0/3377000355 >> v1:172.21.15.160:6800/2859204856 conn(0x7f32f407db60 legacy=0x7f32f4080020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:30.575 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.574+0000 7f330cb0e640 1 -- 172.21.15.160:0/3377000355 >> v1:172.21.15.160:6789/0 conn(0x7f33081065d0 legacy=0x7f3308105a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:30.575 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.574+0000 7f330cb0e640 1 -- 172.21.15.160:0/3377000355 shutdown_connections 2024-09-16T09:04:30.575 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.574+0000 7f330cb0e640 1 -- 172.21.15.160:0/3377000355 >> 172.21.15.160:0/3377000355 conn(0x7f3308079180 msgr2=0x7f3308110f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:30.576 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.574+0000 7f330cb0e640 1 -- 172.21.15.160:0/3377000355 shutdown_connections 2024-09-16T09:04:30.576 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.574+0000 7f330cb0e640 1 -- 172.21.15.160:0/3377000355 wait complete. 2024-09-16T09:04:30.584 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph osd pool delete b3210054-d6e7-4974-b854-ef9506616c26 b3210054-d6e7-4974-b854-ef9506616c26 --yes-i-really-really-mean-it 2024-09-16T09:04:30.666 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.664+0000 7ff19cc16640 1 -- 172.21.15.160:0/3976864815 >> v1:172.21.15.160:6790/0 conn(0x7ff19810a5f0 legacy=0x7ff19810c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:30.667 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.664+0000 7ff19cc16640 1 -- 172.21.15.160:0/3976864815 shutdown_connections 2024-09-16T09:04:30.667 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.664+0000 7ff19cc16640 1 -- 172.21.15.160:0/3976864815 >> 172.21.15.160:0/3976864815 conn(0x7ff1980fd4b0 msgr2=0x7ff1980ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:30.667 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.664+0000 7ff19cc16640 1 -- 172.21.15.160:0/3976864815 shutdown_connections 2024-09-16T09:04:30.667 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.665+0000 7ff19cc16640 1 -- 172.21.15.160:0/3976864815 wait complete. 2024-09-16T09:04:30.667 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.665+0000 7ff19cc16640 1 Processor -- start 2024-09-16T09:04:30.667 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.665+0000 7ff19cc16640 1 -- start start 2024-09-16T09:04:30.670 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.665+0000 7ff19cc16640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff1981a9b00 con 0x7ff19810a5f0 2024-09-16T09:04:30.670 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.665+0000 7ff19cc16640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff1981aad00 con 0x7ff19810e1c0 2024-09-16T09:04:30.670 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.665+0000 7ff19cc16640 1 -- --> v1:172.21.15.160:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff1981abf00 con 0x7ff198101a90 2024-09-16T09:04:30.670 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.665+0000 7ff19659c640 1 --1- >> v1:172.21.15.160:6790/0 conn(0x7ff198101a90 0x7ff19810ffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.160:6790/0 says I am v1:172.21.15.160:43676/0 (socket says 172.21.15.160:43676) 2024-09-16T09:04:30.670 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff19659c640 1 -- 172.21.15.160:0/3376936843 learned_addr learned my addr 172.21.15.160:0/3376936843 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-09-16T09:04:30.670 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2154485184 0 0) 0x7ff1981a9b00 con 0x7ff19810a5f0 2024-09-16T09:04:30.670 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff180003520 con 0x7ff19810a5f0 2024-09-16T09:04:30.670 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 <== mon.2 v1:172.21.15.160:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 718193346 0 0) 0x7ff1981abf00 con 0x7ff198101a90 2024-09-16T09:04:30.670 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 --> v1:172.21.15.160:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1981a9b00 con 0x7ff198101a90 2024-09-16T09:04:30.670 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4182683616 0 0) 0x7ff1981aad00 con 0x7ff19810e1c0 2024-09-16T09:04:30.671 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1981abf00 con 0x7ff19810e1c0 2024-09-16T09:04:30.671 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 124398463 0 0) 0x7ff180003520 con 0x7ff19810a5f0 2024-09-16T09:04:30.671 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 --> v1:172.21.15.160:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff1981aad00 con 0x7ff19810a5f0 2024-09-16T09:04:30.671 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ff18c001650 con 0x7ff19810a5f0 2024-09-16T09:04:30.671 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 <== mon.2 v1:172.21.15.160:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1536404164 0 0) 0x7ff1981a9b00 con 0x7ff198101a90 2024-09-16T09:04:30.671 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 --> v1:172.21.15.160:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff180003520 con 0x7ff198101a90 2024-09-16T09:04:30.671 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 <== mon.2 v1:172.21.15.160:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ff188001690 con 0x7ff198101a90 2024-09-16T09:04:30.671 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 <== mon.0 v1:172.21.15.160:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3572929798 0 0) 0x7ff1981aad00 con 0x7ff19810a5f0 2024-09-16T09:04:30.671 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 >> v1:172.21.15.160:6790/0 conn(0x7ff198101a90 legacy=0x7ff19810ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:30.671 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 >> v1:172.21.15.169:6789/0 conn(0x7ff19810e1c0 legacy=0x7ff1981a8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:30.671 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1981ad1a0 con 0x7ff19810a5f0 2024-09-16T09:04:30.671 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff19cc16640 1 -- 172.21.15.160:0/3376936843 --> v1:172.21.15.160:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff1981ac130 con 0x7ff19810a5f0 2024-09-16T09:04:30.671 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff19cc16640 1 -- 172.21.15.160:0/3376936843 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff1981ac6a0 con 0x7ff19810a5f0 2024-09-16T09:04:30.671 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 <== mon.0 v1:172.21.15.160:6789/0 5 ==== config(24 keys) ==== 1029+0+0 (unknown 3145825662 0 0) 0x7ff18c001aa0 con 0x7ff19810a5f0 2024-09-16T09:04:30.672 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.666+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 <== mon.0 v1:172.21.15.160:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 840597535 0 0) 0x7ff18c004b00 con 0x7ff19810a5f0 2024-09-16T09:04:30.672 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.667+0000 7ff19cc16640 1 -- 172.21.15.160:0/3376936843 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff178005080 con 0x7ff19810a5f0 2024-09-16T09:04:30.672 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.668+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 <== mon.0 v1:172.21.15.160:6789/0 7 ==== mgrmap(e 22) ==== 103103+0+0 (unknown 3900846400 0 0) 0x7ff18c003ea0 con 0x7ff19810a5f0 2024-09-16T09:04:30.672 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.668+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 <== mon.0 v1:172.21.15.160:6789/0 8 ==== osd_map(766..766 src has 1..766) ==== 7814+0+0 (unknown 1412020925 0 0) 0x7ff18c0987e0 con 0x7ff19810a5f0 2024-09-16T09:04:30.672 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.669+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=767}) -- 0x7ff1981aad00 con 0x7ff19810a5f0 2024-09-16T09:04:30.674 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.672+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 <== mon.0 v1:172.21.15.160:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (unknown 1092875540 0 1648304144) 0x7ff18c063780 con 0x7ff19810a5f0 2024-09-16T09:04:30.848 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:30.847+0000 7ff19cc16640 1 -- 172.21.15.160:0/3376936843 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pool2": "b3210054-d6e7-4974-b854-ef9506616c26", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff178005370 con 0x7ff19810a5f0 2024-09-16T09:04:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:30 smithi169 ceph-mon[33465]: pgmap v1257: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:04:30.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:30 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3377000355' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pool2": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:04:30.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:30 smithi160 ceph-mon[30137]: pgmap v1257: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:04:30.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:30 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3377000355' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pool2": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:04:30.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:30 smithi160 ceph-mon[40028]: pgmap v1257: 188 pgs: 188 active+clean; 897 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:04:30.948 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:30 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3377000355' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pool2": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:04:31.468 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:31.466+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 <== mon.0 v1:172.21.15.160:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pool2": "b3210054-d6e7-4974-b854-ef9506616c26", "yes_i_really_really_mean_it": true}]=0 pool 'b3210054-d6e7-4974-b854-ef9506616c26' removed v767) ==== 248+0+0 (unknown 3880309500 0 0) 0x7ff18c0686c0 con 0x7ff19810a5f0 2024-09-16T09:04:31.470 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:31.469+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 <== mon.0 v1:172.21.15.160:6789/0 11 ==== osd_map(767..767 src has 1..767) ==== 296+0+0 (unknown 2455394656 0 0) 0x7ff18c096580 con 0x7ff19810a5f0 2024-09-16T09:04:31.470 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:31.469+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 --> v1:172.21.15.160:6789/0 -- mon_subscribe({osdmap=768}) -- 0x7ff1981abf00 con 0x7ff19810a5f0 2024-09-16T09:04:31.582 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:31.579+0000 7ff19cc16640 1 -- 172.21.15.160:0/3376936843 --> v1:172.21.15.160:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pool2": "b3210054-d6e7-4974-b854-ef9506616c26", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff178005c40 con 0x7ff19810a5f0 2024-09-16T09:04:31.582 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:31.580+0000 7ff1877fe640 1 -- 172.21.15.160:0/3376936843 <== mon.0 v1:172.21.15.160:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pool2": "b3210054-d6e7-4974-b854-ef9506616c26", "yes_i_really_really_mean_it": true}]=0 pool 'b3210054-d6e7-4974-b854-ef9506616c26' does not exist v767) ==== 255+0+0 (unknown 1390248414 0 0) 0x7ff18c05ae80 con 0x7ff19810a5f0 2024-09-16T09:04:31.582 INFO:tasks.workunit.client.0.smithi160.stderr:pool 'b3210054-d6e7-4974-b854-ef9506616c26' does not exist 2024-09-16T09:04:31.584 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:31.582+0000 7ff19cc16640 1 -- 172.21.15.160:0/3376936843 >> v1:172.21.15.160:6800/2859204856 conn(0x7ff18007bb40 legacy=0x7ff18007e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:31.584 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:31.582+0000 7ff19cc16640 1 -- 172.21.15.160:0/3376936843 >> v1:172.21.15.160:6789/0 conn(0x7ff19810a5f0 legacy=0x7ff1981a49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:31.584 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:31.582+0000 7ff19cc16640 1 -- 172.21.15.160:0/3376936843 shutdown_connections 2024-09-16T09:04:31.584 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:31.582+0000 7ff19cc16640 1 -- 172.21.15.160:0/3376936843 >> 172.21.15.160:0/3376936843 conn(0x7ff1980fd4b0 msgr2=0x7ff198107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:31.585 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:31.582+0000 7ff19cc16640 1 -- 172.21.15.160:0/3376936843 shutdown_connections 2024-09-16T09:04:31.585 INFO:tasks.workunit.client.0.smithi160.stderr:2024-09-16T09:04:31.582+0000 7ff19cc16640 1 -- 172.21.15.160:0/3376936843 wait complete. 2024-09-16T09:04:31.593 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo OK 2024-09-16T09:04:31.594 INFO:tasks.workunit.client.0.smithi160.stdout:OK 2024-09-16T09:04:31.595 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T09:04:31.595 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T09:04:31.662 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-09-16T09:04:31.662 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-16T09:04:31.725 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3377000355' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pool2": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:04:31.725 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:31 smithi160 ceph-mon[30137]: osdmap e766: 8 total, 8 up, 8 in 2024-09-16T09:04:31.725 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3377000355' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pool2": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:04:31.725 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:31 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3376936843' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pool2": "b3210054-d6e7-4974-b854-ef9506616c26", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:04:31.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3377000355' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pool2": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:04:31.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:31 smithi160 ceph-mon[40028]: osdmap e766: 8 total, 8 up, 8 in 2024-09-16T09:04:31.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3377000355' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pool2": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:04:31.726 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:31 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3376936843' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pool2": "b3210054-d6e7-4974-b854-ef9506616c26", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:04:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3377000355' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pool2": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:04:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:31 smithi169 ceph-mon[33465]: osdmap e766: 8 total, 8 up, 8 in 2024-09-16T09:04:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3377000355' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "pool2": "0356ba9f-a8cb-465e-96c7-f3ebcc55b3da", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:04:31.850 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:31 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3376936843' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pool2": "b3210054-d6e7-4974-b854-ef9506616c26", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:04:31.985 DEBUG:teuthology.parallel:result is None 2024-09-16T09:04:31.986 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T09:04:32.051 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T09:04:32.051 DEBUG:teuthology.orchestra.run.smithi160:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-16T09:04:32.111 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T09:04:32.111 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-16T09:04:32.120 INFO:tasks.cephadm:Teardown begin 2024-09-16T09:04:32.120 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T09:04:32.176 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T09:04:32.209 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-16T09:04:32.209 DEBUG:teuthology.orchestra.run.smithi160:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T09:04:32.232 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T09:04:32.267 INFO:tasks.cephadm:Stopping all daemons... 2024-09-16T09:04:32.268 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-16T09:04:32.268 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mon.a 2024-09-16T09:04:32.628 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:32 smithi160 systemd[1]: Stopping Ceph mon.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:04:32.628 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:32 smithi160 ceph-mon[30137]: pgmap v1259: 176 pgs: 176 active+clean; 708 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:04:32.628 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:32 smithi160 ceph-mon[30137]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-16T09:04:32.628 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3376936843' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pool2": "b3210054-d6e7-4974-b854-ef9506616c26", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:04:32.628 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:32 smithi160 ceph-mon[30137]: osdmap e767: 8 total, 8 up, 8 in 2024-09-16T09:04:32.628 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:32 smithi160 ceph-mon[30137]: from='client.? v1:172.21.15.160:0/3376936843' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pool2": "b3210054-d6e7-4974-b854-ef9506616c26", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:04:32.629 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:32 smithi160 ceph-mon[40028]: pgmap v1259: 176 pgs: 176 active+clean; 708 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:04:32.629 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:32 smithi160 ceph-mon[40028]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-16T09:04:32.629 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3376936843' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pool2": "b3210054-d6e7-4974-b854-ef9506616c26", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:04:32.629 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:32 smithi160 ceph-mon[40028]: osdmap e767: 8 total, 8 up, 8 in 2024-09-16T09:04:32.629 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:32 smithi160 ceph-mon[40028]: from='client.? v1:172.21.15.160:0/3376936843' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pool2": "b3210054-d6e7-4974-b854-ef9506616c26", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:04:32.739 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:32 smithi169 ceph-mon[33465]: pgmap v1259: 176 pgs: 176 active+clean; 708 KiB data, 801 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:04:32.739 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:32 smithi169 ceph-mon[33465]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-16T09:04:32.739 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3376936843' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pool2": "b3210054-d6e7-4974-b854-ef9506616c26", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:04:32.739 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:32 smithi169 ceph-mon[33465]: osdmap e767: 8 total, 8 up, 8 in 2024-09-16T09:04:32.739 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:32 smithi169 ceph-mon[33465]: from='client.? v1:172.21.15.160:0/3376936843' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "b3210054-d6e7-4974-b854-ef9506616c26", "pool2": "b3210054-d6e7-4974-b854-ef9506616c26", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:04:32.948 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:32 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:09:04:32] ENGINE Bus STOPPING 2024-09-16T09:04:32.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:32 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-a[30114]: 2024-09-16T09:04:32.625+0000 7f790f8a0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:04:32.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:32 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-a[30114]: 2024-09-16T09:04:32.625+0000 7f790f8a0640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-16T09:04:32.948 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:32 smithi160 podman[146805]: 2024-09-16 09:04:32.657572156 +0000 UTC m=+0.195128585 container died a8429aef53bc2ece1ba7d93da44da43b19a5715ead119fdc0a681a6d0110d64a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-16T09:04:33.402 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:33 smithi160 podman[146805]: 2024-09-16 09:04:33.150119372 +0000 UTC m=+0.687675844 container cleanup a8429aef53bc2ece1ba7d93da44da43b19a5715ead119fdc0a681a6d0110d64a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T09:04:33.402 INFO:journalctl@ceph.mon.a.smithi160.stdout:Sep 16 09:04:33 smithi160 bash[146805]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-a 2024-09-16T09:04:33.403 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:33 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:09:04:33] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T09:04:33.403 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:33 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:09:04:33] ENGINE Bus STOPPED 2024-09-16T09:04:33.403 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:33 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:09:04:33] ENGINE Bus STARTING 2024-09-16T09:04:33.403 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:33 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:09:04:33] ENGINE Serving on http://:::9283 2024-09-16T09:04:33.403 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:33 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:09:04:33] ENGINE Bus STARTED 2024-09-16T09:04:33.537 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mon.a.service' 2024-09-16T09:04:33.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:04:33.576 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-16T09:04:33.576 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-16T09:04:33.576 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mon.c 2024-09-16T09:04:33.698 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:33 smithi160 systemd[1]: Stopping Ceph mon.c for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:04:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:33 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-c[40005]: 2024-09-16T09:04:33.898+0000 7f471ef30640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:04:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:33 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-c[40005]: 2024-09-16T09:04:33.898+0000 7f471ef30640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-16T09:04:34.198 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:34 smithi160 podman[146949]: 2024-09-16 09:04:34.082604417 +0000 UTC m=+0.288525644 container died 9e91ddc50d3df5817dbe9fdb9d73061eba4f4dcdae79fefb045f0e1d605c2c2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-c, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-16T09:04:34.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:33 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:09:04:33] ENGINE Bus STOPPING 2024-09-16T09:04:34.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:09:04:34] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T09:04:34.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:09:04:34] ENGINE Bus STOPPED 2024-09-16T09:04:34.198 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:09:04:34] ENGINE Bus STARTING 2024-09-16T09:04:34.473 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:09:04:34] ENGINE Serving on http://:::9283 2024-09-16T09:04:34.473 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: [16/Sep/2024:09:04:34] ENGINE Bus STARTED 2024-09-16T09:04:34.766 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:34 smithi160 podman[146949]: 2024-09-16 09:04:34.472843645 +0000 UTC m=+0.678764871 container cleanup 9e91ddc50d3df5817dbe9fdb9d73061eba4f4dcdae79fefb045f0e1d605c2c2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-c, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T09:04:34.766 INFO:journalctl@ceph.mon.c.smithi160.stdout:Sep 16 09:04:34 smithi160 bash[146949]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-c 2024-09-16T09:04:34.894 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mon.c.service' 2024-09-16T09:04:34.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:04:34.930 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-16T09:04:34.930 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-16T09:04:34.930 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mon.b 2024-09-16T09:04:35.197 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:34 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: ::ffff:172.21.15.169 - - [16/Sep/2024:09:04:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:04:35.238 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:34 smithi169 systemd[1]: Stopping Ceph mon.b for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:04:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-b[33442]: 2024-09-16T09:04:35.236+0000 7ffac5913640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:04:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-b[33442]: 2024-09-16T09:04:35.236+0000 7ffac5913640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-16T09:04:35.600 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:35 smithi169 podman[68130]: 2024-09-16 09:04:35.461091552 +0000 UTC m=+0.335880949 container died 4b357747a229cec57221a812a464350639522901b2c397f7c81db0008a17f211 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2) 2024-09-16T09:04:36.100 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:04:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug there is no tcmu-runner data available 2024-09-16T09:04:36.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:35 smithi169 podman[68130]: 2024-09-16 09:04:35.968999892 +0000 UTC m=+0.843789325 container cleanup 4b357747a229cec57221a812a464350639522901b2c397f7c81db0008a17f211 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T09:04:36.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Sep 16 09:04:35 smithi169 bash[68130]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mon-b 2024-09-16T09:04:36.310 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mon.b.service' 2024-09-16T09:04:36.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:04:36.359 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-16T09:04:36.360 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-16T09:04:36.360 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mgr.y 2024-09-16T09:04:36.663 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:36 smithi160 systemd[1]: Stopping Ceph mgr.y for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:04:36.948 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:36 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: 2024-09-16T09:04:36.662+0000 7f83ec022640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:04:36.948 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:36 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y[30399]: 2024-09-16T09:04:36.662+0000 7f83ec022640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-16T09:04:36.948 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:36 smithi160 podman[147094]: 2024-09-16 09:04:36.693819655 +0000 UTC m=+0.147432149 container died 2dab0971fcc381fe821a07e377e243b621675202314f722d7bc826dc411b76ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-16T09:04:37.396 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:37 smithi160 podman[147094]: 2024-09-16 09:04:37.085989737 +0000 UTC m=+0.539602228 container cleanup 2dab0971fcc381fe821a07e377e243b621675202314f722d7bc826dc411b76ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-16T09:04:37.397 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Sep 16 09:04:37 smithi160 bash[147094]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-y 2024-09-16T09:04:37.561 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-16T09:04:37.596 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:04:37.596 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-16T09:04:37.596 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-16T09:04:37.596 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mgr.x 2024-09-16T09:04:37.932 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 09:04:37 smithi169 systemd[1]: Stopping Ceph mgr.x for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:04:38.216 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 09:04:37 smithi169 podman[68261]: 2024-09-16 09:04:37.931924614 +0000 UTC m=+0.131532919 container died 3c5a1d4782bb2e7da7bc10be489ac52129ab757f0004192d3778925ff2c98c1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T09:04:38.576 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 09:04:38 smithi169 podman[68261]: 2024-09-16 09:04:38.315343424 +0000 UTC m=+0.514951730 container cleanup 3c5a1d4782bb2e7da7bc10be489ac52129ab757f0004192d3778925ff2c98c1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:04:38.576 INFO:journalctl@ceph.mgr.x.smithi169.stdout:Sep 16 09:04:38 smithi169 bash[68261]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-mgr-x 2024-09-16T09:04:38.703 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-16T09:04:38.739 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:04:38.739 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-16T09:04:38.739 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-16T09:04:38.740 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.0 2024-09-16T09:04:39.051 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 09:04:38 smithi160 systemd[1]: Stopping Ceph osd.0 for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:04:39.447 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 09:04:39 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-0[43659]: 2024-09-16T09:04:39.050+0000 7fa6a8031640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:04:39.448 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 09:04:39 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-0[43659]: 2024-09-16T09:04:39.050+0000 7fa6a8031640 -1 osd.0 767 *** Got signal Terminated *** 2024-09-16T09:04:39.448 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 09:04:39 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-0[43659]: 2024-09-16T09:04:39.050+0000 7fa6a8031640 -1 osd.0 767 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:04:44.698 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 09:04:44 smithi160 podman[147226]: 2024-09-16 09:04:44.434212 +0000 UTC m=+5.510408094 container died 155d7dc14825e1a4cb3bb3df8b354e0a1b5d8125fbd7ab542cfd6c300089bffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T09:04:45.041 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 09:04:44 smithi160 podman[147226]: 2024-09-16 09:04:44.815341402 +0000 UTC m=+5.891537498 container cleanup 155d7dc14825e1a4cb3bb3df8b354e0a1b5d8125fbd7ab542cfd6c300089bffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-16T09:04:45.042 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 09:04:44 smithi160 bash[147226]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-0 2024-09-16T09:04:45.375 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 09:04:45 smithi160 podman[147247]: 2024-09-16 09:04:45.041435483 +0000 UTC m=+0.605369337 container remove 155d7dc14825e1a4cb3bb3df8b354e0a1b5d8125fbd7ab542cfd6c300089bffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=) 2024-09-16T09:04:45.695 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 09:04:45 smithi160 podman[147342]: 2024-09-16 09:04:45.375659987 +0000 UTC m=+0.111088980 container create f9b25d7eaf359c6e1aca0536ac352475eefb314e0e3a276fcf03b39d7f9ffc28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T09:04:45.695 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 09:04:45 smithi160 podman[147342]: 2024-09-16 09:04:45.280272506 +0000 UTC m=+0.015701509 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:04:45.696 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 09:04:45 smithi160 podman[147342]: 2024-09-16 09:04:45.595389719 +0000 UTC m=+0.330818721 container init f9b25d7eaf359c6e1aca0536ac352475eefb314e0e3a276fcf03b39d7f9ffc28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-16T09:04:45.696 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 09:04:45 smithi160 podman[147342]: 2024-09-16 09:04:45.600210177 +0000 UTC m=+0.335639172 container start f9b25d7eaf359c6e1aca0536ac352475eefb314e0e3a276fcf03b39d7f9ffc28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T09:04:45.696 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 09:04:45 smithi160 podman[147342]: 2024-09-16 09:04:45.69520668 +0000 UTC m=+0.430635700 container attach f9b25d7eaf359c6e1aca0536ac352475eefb314e0e3a276fcf03b39d7f9ffc28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2) 2024-09-16T09:04:45.948 INFO:journalctl@ceph.osd.0.smithi160.stdout:Sep 16 09:04:45 smithi160 podman[147342]: 2024-09-16 09:04:45.792709298 +0000 UTC m=+0.528138296 container died f9b25d7eaf359c6e1aca0536ac352475eefb314e0e3a276fcf03b39d7f9ffc28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-16T09:04:46.397 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.0.service' 2024-09-16T09:04:46.436 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:04:46.437 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-16T09:04:46.437 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-16T09:04:46.437 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.1 2024-09-16T09:04:46.698 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 09:04:46 smithi160 systemd[1]: Stopping Ceph osd.1 for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:04:47.197 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 09:04:46 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-1[49686]: 2024-09-16T09:04:46.770+0000 7f670a886640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:04:47.198 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 09:04:46 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-1[49686]: 2024-09-16T09:04:46.770+0000 7f670a886640 -1 osd.1 767 *** Got signal Terminated *** 2024-09-16T09:04:47.198 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 09:04:46 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-1[49686]: 2024-09-16T09:04:46.770+0000 7f670a886640 -1 osd.1 767 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:04:52.594 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 09:04:52 smithi160 podman[147471]: 2024-09-16 09:04:52.29129128 +0000 UTC m=+5.636855072 container died 232d8227b381503aa897efdddbbcb28acbca8153da98b8b58ed04eae52aa70c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:04:52.948 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 09:04:52 smithi160 podman[147471]: 2024-09-16 09:04:52.709532364 +0000 UTC m=+6.055096153 container cleanup 232d8227b381503aa897efdddbbcb28acbca8153da98b8b58ed04eae52aa70c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-1, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-16T09:04:52.948 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 09:04:52 smithi160 bash[147471]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-1 2024-09-16T09:04:53.447 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 09:04:53 smithi160 podman[147494]: 2024-09-16 09:04:53.002683009 +0000 UTC m=+0.700864771 container remove 232d8227b381503aa897efdddbbcb28acbca8153da98b8b58ed04eae52aa70c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True) 2024-09-16T09:04:53.742 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 09:04:53 smithi160 podman[147587]: 2024-09-16 09:04:53.381976797 +0000 UTC m=+0.015805650 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:04:53.742 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 09:04:53 smithi160 podman[147587]: 2024-09-16 09:04:53.488802616 +0000 UTC m=+0.122631464 container create 5aa8925a5bb12f363043ebf738ba069bb7895c9d4b80480ebe165cc23bc11a03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:04:53.742 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 09:04:53 smithi160 podman[147587]: 2024-09-16 09:04:53.741593014 +0000 UTC m=+0.375421876 container init 5aa8925a5bb12f363043ebf738ba069bb7895c9d4b80480ebe165cc23bc11a03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T09:04:54.198 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 09:04:53 smithi160 podman[147587]: 2024-09-16 09:04:53.747045021 +0000 UTC m=+0.380873870 container start 5aa8925a5bb12f363043ebf738ba069bb7895c9d4b80480ebe165cc23bc11a03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:04:54.198 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 09:04:53 smithi160 podman[147587]: 2024-09-16 09:04:53.849562425 +0000 UTC m=+0.483391285 container attach 5aa8925a5bb12f363043ebf738ba069bb7895c9d4b80480ebe165cc23bc11a03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True) 2024-09-16T09:04:54.198 INFO:journalctl@ceph.osd.1.smithi160.stdout:Sep 16 09:04:53 smithi160 podman[147587]: 2024-09-16 09:04:53.931669732 +0000 UTC m=+0.565498588 container died 5aa8925a5bb12f363043ebf738ba069bb7895c9d4b80480ebe165cc23bc11a03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-09-16T09:04:54.510 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.1.service' 2024-09-16T09:04:54.548 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:04:54.548 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-16T09:04:54.549 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T09:04:54.549 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.2 2024-09-16T09:04:54.884 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 09:04:54 smithi160 systemd[1]: Stopping Ceph osd.2 for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:04:55.197 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 09:04:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-2[55873]: 2024-09-16T09:04:54.883+0000 7f1fd2101640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:04:55.198 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 09:04:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-2[55873]: 2024-09-16T09:04:54.883+0000 7f1fd2101640 -1 osd.2 767 *** Got signal Terminated *** 2024-09-16T09:04:55.198 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 09:04:54 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-2[55873]: 2024-09-16T09:04:54.883+0000 7f1fd2101640 -1 osd.2 767 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:04:57.447 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:04:57 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:04:57.018Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.160:8443: connect: connection refused" 2024-09-16T09:04:57.448 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:04:57 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:04:57.018Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.169:8443: connect: connection refused" 2024-09-16T09:05:00.475 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 09:05:00 smithi160 podman[147713]: 2024-09-16 09:05:00.210387623 +0000 UTC m=+5.432670954 container died 08f6d0ee2067965e02376e9b2d4f292b178228dee3ec205a3664c0cf3af1e613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-16T09:05:00.749 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 09:05:00 smithi160 podman[147713]: 2024-09-16 09:05:00.556261108 +0000 UTC m=+5.778544441 container cleanup 08f6d0ee2067965e02376e9b2d4f292b178228dee3ec205a3664c0cf3af1e613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T09:05:00.749 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 09:05:00 smithi160 bash[147713]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-2 2024-09-16T09:05:01.169 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 09:05:00 smithi160 podman[147733]: 2024-09-16 09:05:00.749124563 +0000 UTC m=+0.535020952 container remove 08f6d0ee2067965e02376e9b2d4f292b178228dee3ec205a3664c0cf3af1e613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T09:05:01.427 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 09:05:01 smithi160 podman[147830]: 2024-09-16 09:05:01.071730071 +0000 UTC m=+0.015544751 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:05:01.427 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 09:05:01 smithi160 podman[147830]: 2024-09-16 09:05:01.191542832 +0000 UTC m=+0.135357508 container create cab421ad2ff7933fc6d34a982eb6382649f70151739e97aee681b13a5335f385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T09:05:01.427 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 09:05:01 smithi160 podman[147830]: 2024-09-16 09:05:01.427126019 +0000 UTC m=+0.370940701 container init cab421ad2ff7933fc6d34a982eb6382649f70151739e97aee681b13a5335f385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T09:05:01.698 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 09:05:01 smithi160 podman[147830]: 2024-09-16 09:05:01.432024211 +0000 UTC m=+0.375838889 container start cab421ad2ff7933fc6d34a982eb6382649f70151739e97aee681b13a5335f385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T09:05:01.698 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 09:05:01 smithi160 podman[147830]: 2024-09-16 09:05:01.518588461 +0000 UTC m=+0.462403150 container attach cab421ad2ff7933fc6d34a982eb6382649f70151739e97aee681b13a5335f385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-16T09:05:01.698 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 09:05:01 smithi160 podman[147830]: 2024-09-16 09:05:01.622613656 +0000 UTC m=+0.566428338 container died cab421ad2ff7933fc6d34a982eb6382649f70151739e97aee681b13a5335f385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T09:05:02.182 INFO:journalctl@ceph.osd.2.smithi160.stdout:Sep 16 09:05:02 smithi160 podman[147873]: 2024-09-16 09:05:02.180349391 +0000 UTC m=+0.551214788 container remove cab421ad2ff7933fc6d34a982eb6382649f70151739e97aee681b13a5335f385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-2-deactivate, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T09:05:02.204 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T09:05:02.242 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:05:02.242 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T09:05:02.242 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-16T09:05:02.242 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.3 2024-09-16T09:05:02.594 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 09:05:02 smithi160 systemd[1]: Stopping Ceph osd.3 for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:05:02.947 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 09:05:02 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-3[61676]: 2024-09-16T09:05:02.593+0000 7f72ce6e9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:05:02.948 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 09:05:02 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-3[61676]: 2024-09-16T09:05:02.593+0000 7f72ce6e9640 -1 osd.3 767 *** Got signal Terminated *** 2024-09-16T09:05:02.948 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 09:05:02 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-3[61676]: 2024-09-16T09:05:02.593+0000 7f72ce6e9640 -1 osd.3 767 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:05:05.370 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:05 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:05.115+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:06.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:05 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:05.652+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:06.599 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:06 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:06.123+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:07.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:06 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:06.680+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:07.448 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:07 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:05:07.018Z caller=dispatch.go:353 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"\": dial tcp 172.21.15.160:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"\": dial tcp 172.21.15.169:8443: connect: connection refused" 2024-09-16T09:05:07.448 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:07 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:05:07.018Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.160:8443: connect: connection refused" 2024-09-16T09:05:07.448 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:07 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:05:07.019Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.169:8443: connect: connection refused" 2024-09-16T09:05:07.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:07 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:07.119+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:08.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:07 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:07.659+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:08.100 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:07 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4[38096]: 2024-09-16T09:05:07.891+0000 7fee3b7f4640 -1 osd.4 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:43.522646+0000 front 2024-09-16T09:04:43.522422+0000 (oldest deadline 2024-09-16T09:05:07.022515+0000) 2024-09-16T09:05:08.367 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 09:05:08 smithi160 podman[147956]: 2024-09-16 09:05:08.088608486 +0000 UTC m=+5.622321115 container died 14e70006e47c7a5455231efd75106ec7b538ad7e51b2247ee2e58524bf40e091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:05:08.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:08.162+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:08.600 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:08.237Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.160:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.160:8765: connect: connection refused" 2024-09-16T09:05:08.600 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:08.241Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=smb msg="Unable to refresh target groups" err="Get \"http://172.21.15.160:8765/sd/prometheus/sd-config?service=smb\": dial tcp 172.21.15.160:8765: connect: connection refused" 2024-09-16T09:05:08.600 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:08.241Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.160:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.160:8765: connect: connection refused" 2024-09-16T09:05:08.600 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:08.241Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.160:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.160:8765: connect: connection refused" 2024-09-16T09:05:08.600 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:08.242Z caller=refresh.go:90 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.160:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.160:8765: connect: connection refused" 2024-09-16T09:05:08.601 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:08.243Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.160:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.160:8765: connect: connection refused" 2024-09-16T09:05:08.601 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:08.243Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nfs msg="Unable to refresh target groups" err="Get \"http://172.21.15.160:8765/sd/prometheus/sd-config?service=nfs\": dial tcp 172.21.15.160:8765: connect: connection refused" 2024-09-16T09:05:08.697 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 09:05:08 smithi160 podman[147956]: 2024-09-16 09:05:08.46635164 +0000 UTC m=+6.000064268 container cleanup 14e70006e47c7a5455231efd75106ec7b538ad7e51b2247ee2e58524bf40e091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-3, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-16T09:05:08.698 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 09:05:08 smithi160 bash[147956]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-3 2024-09-16T09:05:09.075 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 09:05:08 smithi160 podman[147985]: 2024-09-16 09:05:08.742574637 +0000 UTC m=+0.645944552 container remove 14e70006e47c7a5455231efd75106ec7b538ad7e51b2247ee2e58524bf40e091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-16T09:05:09.100 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4[38096]: 2024-09-16T09:05:08.856+0000 7fee3b7f4640 -1 osd.4 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:43.522646+0000 front 2024-09-16T09:04:43.522422+0000 (oldest deadline 2024-09-16T09:05:07.022515+0000) 2024-09-16T09:05:09.101 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:08.798+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:09.101 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:08 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:08.665+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:09.326 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 09:05:09 smithi160 podman[148082]: 2024-09-16 09:05:08.978190974 +0000 UTC m=+0.015919741 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:05:09.327 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 09:05:09 smithi160 podman[148082]: 2024-09-16 09:05:09.118461455 +0000 UTC m=+0.156190208 container create 090f52e06d3dc12bc6420fb863a8d8a0ff5f61cf3fef59d804e895b472ef4029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T09:05:09.327 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 09:05:09 smithi160 podman[148082]: 2024-09-16 09:05:09.320891074 +0000 UTC m=+0.358619829 container init 090f52e06d3dc12bc6420fb863a8d8a0ff5f61cf3fef59d804e895b472ef4029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-09-16T09:05:09.327 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 09:05:09 smithi160 podman[148082]: 2024-09-16 09:05:09.326201558 +0000 UTC m=+0.363930321 container start 090f52e06d3dc12bc6420fb863a8d8a0ff5f61cf3fef59d804e895b472ef4029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T09:05:09.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:09 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:09.157+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:09.698 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 09:05:09 smithi160 podman[148082]: 2024-09-16 09:05:09.404089096 +0000 UTC m=+0.441817864 container attach 090f52e06d3dc12bc6420fb863a8d8a0ff5f61cf3fef59d804e895b472ef4029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-3-deactivate, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:05:09.698 INFO:journalctl@ceph.osd.3.smithi160.stdout:Sep 16 09:05:09 smithi160 podman[148082]: 2024-09-16 09:05:09.511307605 +0000 UTC m=+0.549036372 container died 090f52e06d3dc12bc6420fb863a8d8a0ff5f61cf3fef59d804e895b472ef4029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-16T09:05:10.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:09 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:09.783+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:10.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:09 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:09.701+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:10.100 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:09 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4[38096]: 2024-09-16T09:05:09.891+0000 7fee3b7f4640 -1 osd.4 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:43.522646+0000 front 2024-09-16T09:04:43.522422+0000 (oldest deadline 2024-09-16T09:05:07.022515+0000) 2024-09-16T09:05:10.163 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.3.service' 2024-09-16T09:05:10.203 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:05:10.204 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-16T09:05:10.204 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-16T09:05:10.204 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.4 2024-09-16T09:05:10.511 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:10 smithi169 systemd[1]: Stopping Ceph osd.4 for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:05:10.511 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:10 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:10.205+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:10.811 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:10 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:10.736+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:10.811 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:10 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4[38096]: 2024-09-16T09:05:10.510+0000 7fee431e7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:05:10.811 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:10 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4[38096]: 2024-09-16T09:05:10.510+0000 7fee431e7640 -1 osd.4 767 *** Got signal Terminated *** 2024-09-16T09:05:10.811 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:10 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4[38096]: 2024-09-16T09:05:10.510+0000 7fee431e7640 -1 osd.4 767 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:05:11.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:10 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:10.810+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:11.100 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:10 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4[38096]: 2024-09-16T09:05:10.902+0000 7fee3b7f4640 -1 osd.4 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:43.522646+0000 front 2024-09-16T09:04:43.522422+0000 (oldest deadline 2024-09-16T09:05:07.022515+0000) 2024-09-16T09:05:11.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:11 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:11.172+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:12.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:11 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:11.859+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:12.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:11 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:11.735+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:12.101 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:11 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4[38096]: 2024-09-16T09:05:11.882+0000 7fee3b7f4640 -1 osd.4 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:43.522646+0000 front 2024-09-16T09:04:43.522422+0000 (oldest deadline 2024-09-16T09:05:07.022515+0000) 2024-09-16T09:05:12.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:12 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:12.128+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:13.100 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:12 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4[38096]: 2024-09-16T09:05:12.889+0000 7fee3b7f4640 -1 osd.4 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:43.522646+0000 front 2024-09-16T09:04:43.522422+0000 (oldest deadline 2024-09-16T09:05:07.022515+0000) 2024-09-16T09:05:13.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:12 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:12.839+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:13.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:12 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:12.756+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:13.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:13 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:13.103+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:13.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:13 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:13.103+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.694709+0000 front 2024-09-16T09:04:47.694781+0000 (oldest deadline 2024-09-16T09:05:12.394908+0000) 2024-09-16T09:05:14.100 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:13 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4[38096]: 2024-09-16T09:05:13.908+0000 7fee3b7f4640 -1 osd.4 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:43.522646+0000 front 2024-09-16T09:04:43.522422+0000 (oldest deadline 2024-09-16T09:05:07.022515+0000) 2024-09-16T09:05:14.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:13 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:13.838+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:14.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:13 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:13.724+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:14.101 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:13 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:13.724+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:14.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:14 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:14.115+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:14.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:14 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:14.115+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.694709+0000 front 2024-09-16T09:04:47.694781+0000 (oldest deadline 2024-09-16T09:05:12.394908+0000) 2024-09-16T09:05:15.068 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:14 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4[38096]: 2024-09-16T09:05:14.955+0000 7fee3b7f4640 -1 osd.4 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:43.522646+0000 front 2024-09-16T09:04:43.522422+0000 (oldest deadline 2024-09-16T09:05:07.022515+0000) 2024-09-16T09:05:15.068 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:14 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:14.883+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:15.068 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:14 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:14.883+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:15.069 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:14 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:14.771+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:15.069 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:14 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:14.771+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:15.350 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:15.066+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:15.350 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:15.066+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.694709+0000 front 2024-09-16T09:04:47.694781+0000 (oldest deadline 2024-09-16T09:05:12.394908+0000) 2024-09-16T09:05:16.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:15.859+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:16.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:15.859+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:16.101 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:15.793+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:16.101 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:15 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:15.793+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:16.101 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:15 smithi169 podman[68411]: 2024-09-16 09:05:15.85836959 +0000 UTC m=+5.451814054 container died a659fa181a05e67d014b020c55d4664058303bcb655b0501a833842e27c5a478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:05:16.485 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:16 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:16.104+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:16.485 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:16 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:16.105+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.694709+0000 front 2024-09-16T09:04:47.694781+0000 (oldest deadline 2024-09-16T09:05:12.394908+0000) 2024-09-16T09:05:16.486 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:16 smithi169 podman[68411]: 2024-09-16 09:05:16.233613727 +0000 UTC m=+5.827058192 container cleanup a659fa181a05e67d014b020c55d4664058303bcb655b0501a833842e27c5a478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T09:05:16.486 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:16 smithi169 bash[68411]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4 2024-09-16T09:05:16.757 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:16 smithi169 podman[68433]: 2024-09-16 09:05:16.485008997 +0000 UTC m=+0.617317823 container remove a659fa181a05e67d014b020c55d4664058303bcb655b0501a833842e27c5a478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T09:05:17.055 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:16 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:16.849+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:17.056 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:16 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:16.849+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:17.056 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:16 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:16.755+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:17.056 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:16 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:16.755+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:17.056 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:16 smithi169 podman[68530]: 2024-09-16 09:05:16.702317571 +0000 UTC m=+0.017570462 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:05:17.056 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:16 smithi169 podman[68530]: 2024-09-16 09:05:16.811203155 +0000 UTC m=+0.126456078 container create 68980886f3d31d8bab5774dc5d82cf2ee4842a5417cb59050baf89977c777f87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:05:17.350 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:17 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:17.059+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:17.350 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:17 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:17.059+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.694709+0000 front 2024-09-16T09:04:47.694781+0000 (oldest deadline 2024-09-16T09:05:12.394908+0000) 2024-09-16T09:05:17.350 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:17 smithi169 podman[68530]: 2024-09-16 09:05:17.055380643 +0000 UTC m=+0.370633520 container init 68980886f3d31d8bab5774dc5d82cf2ee4842a5417cb59050baf89977c777f87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:05:17.350 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:17 smithi169 podman[68530]: 2024-09-16 09:05:17.060127676 +0000 UTC m=+0.375380553 container start 68980886f3d31d8bab5774dc5d82cf2ee4842a5417cb59050baf89977c777f87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-16T09:05:17.350 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:17 smithi169 podman[68530]: 2024-09-16 09:05:17.138725738 +0000 UTC m=+0.453978616 container attach 68980886f3d31d8bab5774dc5d82cf2ee4842a5417cb59050baf89977c777f87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T09:05:17.351 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 09:05:17 smithi169 podman[68530]: 2024-09-16 09:05:17.248157969 +0000 UTC m=+0.563410862 container died 68980886f3d31d8bab5774dc5d82cf2ee4842a5417cb59050baf89977c777f87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True) 2024-09-16T09:05:17.448 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:17 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:05:17.019Z caller=dispatch.go:353 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.169:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"\": dial tcp 172.21.15.160:8443: connect: connection refused" 2024-09-16T09:05:17.448 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:17 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:05:17.020Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.160:8443: connect: connection refused" 2024-09-16T09:05:17.448 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:17 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:05:17.021Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.169:8443: connect: connection refused" 2024-09-16T09:05:17.834 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:17 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:17.779+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:17.834 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:17 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:17.779+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:17.834 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:17 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:17.800+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:17.835 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:17 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:17.800+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:17.852 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.4.service' 2024-09-16T09:05:17.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:05:17.894 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-16T09:05:17.894 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-16T09:05:17.894 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.5 2024-09-16T09:05:18.100 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:17 smithi169 systemd[1]: Stopping Ceph osd.5 for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:05:18.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:18 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:18.103+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:18.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:18 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:18.103+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.694709+0000 front 2024-09-16T09:04:47.694781+0000 (oldest deadline 2024-09-16T09:05:12.394908+0000) 2024-09-16T09:05:18.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:18 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:18.213+0000 7f2a57b8b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:05:18.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:18 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:18.213+0000 7f2a57b8b640 -1 osd.5 767 *** Got signal Terminated *** 2024-09-16T09:05:18.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:18 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:18.213+0000 7f2a57b8b640 -1 osd.5 767 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:05:19.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:18 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:18.848+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:19.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:18 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:18.848+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:19.101 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:18 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:18.793+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:19.101 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:18 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:18.793+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:19.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:19 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:19.114+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:19.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:19 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:19.114+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.694709+0000 front 2024-09-16T09:04:47.694781+0000 (oldest deadline 2024-09-16T09:05:12.394908+0000) 2024-09-16T09:05:20.093 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:19 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:19.751+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:20.093 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:19 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:19.751+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:20.094 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:19 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:19.885+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:20.094 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:19 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:19.885+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:20.350 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:20 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:20.092+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:20.350 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:20 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:20.092+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.694709+0000 front 2024-09-16T09:04:47.694781+0000 (oldest deadline 2024-09-16T09:05:12.394908+0000) 2024-09-16T09:05:21.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:20 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:20.787+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:21.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:20 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:20.787+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:21.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:20 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:20.787+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:21.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:20 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:20.840+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:21.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:20 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:20.840+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:21.101 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:20 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:20.840+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:57.978802+0000 front 2024-09-16T09:04:57.978701+0000 (oldest deadline 2024-09-16T09:05:20.278576+0000) 2024-09-16T09:05:21.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:21 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:21.142+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:21.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:21 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:21.142+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.694709+0000 front 2024-09-16T09:04:47.694781+0000 (oldest deadline 2024-09-16T09:05:12.394908+0000) 2024-09-16T09:05:22.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:21 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:21.812+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:22.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:21 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:21.812+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:22.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:21 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:21.812+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:22.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:21 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:21.823+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:22.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:21 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:21.823+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:22.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:21 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:21.823+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:57.978802+0000 front 2024-09-16T09:04:57.978701+0000 (oldest deadline 2024-09-16T09:05:20.278576+0000) 2024-09-16T09:05:22.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:22 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:22.114+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:22.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:22 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:22.114+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.694709+0000 front 2024-09-16T09:04:47.694781+0000 (oldest deadline 2024-09-16T09:05:12.394908+0000) 2024-09-16T09:05:22.600 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:22 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:22.114+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:57.496721+0000 front 2024-09-16T09:04:57.496707+0000 (oldest deadline 2024-09-16T09:05:21.596585+0000) 2024-09-16T09:05:23.085 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:22 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:22.817+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:23.085 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:22 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:22.817+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:23.085 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:22 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:22.817+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:23.085 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:22 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:22.848+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:23.086 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:22 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:22.848+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:23.086 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:22 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:22.848+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:57.978802+0000 front 2024-09-16T09:04:57.978701+0000 (oldest deadline 2024-09-16T09:05:20.278576+0000) 2024-09-16T09:05:23.350 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:23 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:23.084+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:38.893847+0000 front 2024-09-16T09:04:38.893729+0000 (oldest deadline 2024-09-16T09:05:04.793869+0000) 2024-09-16T09:05:23.350 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:23 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:23.084+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.694709+0000 front 2024-09-16T09:04:47.694781+0000 (oldest deadline 2024-09-16T09:05:12.394908+0000) 2024-09-16T09:05:23.350 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:23 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5[43717]: 2024-09-16T09:05:23.084+0000 7f2a541a0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:57.496721+0000 front 2024-09-16T09:04:57.496707+0000 (oldest deadline 2024-09-16T09:05:21.596585+0000) 2024-09-16T09:05:23.814 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:23 smithi169 podman[68657]: 2024-09-16 09:05:23.559668414 +0000 UTC m=+5.442943751 container died d1e8a2d16acc9f0dedd8b809e54fdba1ed8ffa1eaeb21578495bcf6e89e998d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-16T09:05:23.814 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:23 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:23.789+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:23.814 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:23 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:23.789+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:23.814 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:23 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:23.789+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:23.814 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:23 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:23.789+0000 7f51f2190640 -1 osd.7 767 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.34478.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e767) 2024-09-16T09:05:24.072 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:23 smithi169 podman[68657]: 2024-09-16 09:05:23.89504417 +0000 UTC m=+5.778319492 container cleanup d1e8a2d16acc9f0dedd8b809e54fdba1ed8ffa1eaeb21578495bcf6e89e998d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-16T09:05:24.072 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:23 smithi169 bash[68657]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5 2024-09-16T09:05:24.072 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:23 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:23.813+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:24.072 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:23 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:23.813+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:24.072 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:23 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:23.813+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:57.978802+0000 front 2024-09-16T09:04:57.978701+0000 (oldest deadline 2024-09-16T09:05:20.278576+0000) 2024-09-16T09:05:24.349 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:24 smithi169 podman[68678]: 2024-09-16 09:05:24.071410298 +0000 UTC m=+0.507729246 container remove d1e8a2d16acc9f0dedd8b809e54fdba1ed8ffa1eaeb21578495bcf6e89e998d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T09:05:24.808 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:24 smithi169 podman[68776]: 2024-09-16 09:05:24.41469281 +0000 UTC m=+0.015980381 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:05:24.808 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:24 smithi169 podman[68776]: 2024-09-16 09:05:24.522151637 +0000 UTC m=+0.123439199 container create 9c5e8e933d6ba380dd9517ab35544c77a55e861451abe86eb3b42e2f95d08737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:05:24.808 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:24 smithi169 podman[68776]: 2024-09-16 09:05:24.71590532 +0000 UTC m=+0.317192892 container init 9c5e8e933d6ba380dd9517ab35544c77a55e861451abe86eb3b42e2f95d08737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T09:05:24.808 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:24 smithi169 podman[68776]: 2024-09-16 09:05:24.721181164 +0000 UTC m=+0.322468727 container start 9c5e8e933d6ba380dd9517ab35544c77a55e861451abe86eb3b42e2f95d08737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-16T09:05:24.808 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:24 smithi169 podman[68776]: 2024-09-16 09:05:24.807417532 +0000 UTC m=+0.408705106 container attach 9c5e8e933d6ba380dd9517ab35544c77a55e861451abe86eb3b42e2f95d08737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True) 2024-09-16T09:05:25.100 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 09:05:24 smithi169 podman[68776]: 2024-09-16 09:05:24.915103768 +0000 UTC m=+0.516391332 container died 9c5e8e933d6ba380dd9517ab35544c77a55e861451abe86eb3b42e2f95d08737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-16T09:05:25.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:24 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:24.824+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:25.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:24 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:24.825+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:25.100 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:24 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:24.825+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:57.978802+0000 front 2024-09-16T09:04:57.978701+0000 (oldest deadline 2024-09-16T09:05:20.278576+0000) 2024-09-16T09:05:25.101 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:24 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:24.828+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:25.101 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:24 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:24.828+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:25.101 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:24 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:24.828+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:25.101 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:24 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:24.828+0000 7f51f2190640 -1 osd.7 767 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.34478.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e767) 2024-09-16T09:05:25.495 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.5.service' 2024-09-16T09:05:25.536 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:05:25.536 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-16T09:05:25.536 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-16T09:05:25.536 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.6 2024-09-16T09:05:25.850 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:25 smithi169 systemd[1]: Stopping Ceph osd.6 for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:05:26.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:25.850+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:26.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:25.850+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:26.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:25.850+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:57.978802+0000 front 2024-09-16T09:04:57.978701+0000 (oldest deadline 2024-09-16T09:05:20.278576+0000) 2024-09-16T09:05:26.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:25.870+0000 7ff8fb08a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:05:26.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:25.870+0000 7ff8fb08a640 -1 osd.6 767 *** Got signal Terminated *** 2024-09-16T09:05:26.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:25.870+0000 7ff8fb08a640 -1 osd.6 767 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:05:26.351 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:25.873+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:26.351 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:25.873+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:26.351 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:25.873+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:26.351 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:25 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:25.873+0000 7f51f2190640 -1 osd.7 767 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.34478.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e767) 2024-09-16T09:05:27.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:26 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:26.863+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:27.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:26 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:26.863+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:27.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:26 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:26.863+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:57.978802+0000 front 2024-09-16T09:04:57.978701+0000 (oldest deadline 2024-09-16T09:05:20.278576+0000) 2024-09-16T09:05:27.350 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:26 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:26.867+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:27.351 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:26 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:26.867+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:27.351 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:26 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:26.867+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:27.351 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:26 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:26.867+0000 7f51f2190640 -1 osd.7 767 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.34478.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e767) 2024-09-16T09:05:27.448 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:05:27.020Z caller=dispatch.go:353 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.160:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.169:8443: connect: connection refused" 2024-09-16T09:05:27.448 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:05:27.021Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.160:8443: connect: connection refused" 2024-09-16T09:05:27.448 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:27 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:05:27.021Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.169:8443: connect: connection refused" 2024-09-16T09:05:28.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:27 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:27.884+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:28.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:27 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:27.884+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:28.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:27 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:27.884+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:57.978802+0000 front 2024-09-16T09:04:57.978701+0000 (oldest deadline 2024-09-16T09:05:20.278576+0000) 2024-09-16T09:05:28.350 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:27 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:27.900+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:28.350 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:27 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:27.900+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:28.351 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:27 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:27.900+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:28.351 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:27 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:27.900+0000 7f51f2190640 -1 osd.7 767 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.34478.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e767) 2024-09-16T09:05:29.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:28 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:28.871+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:29.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:28 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:28.871+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:29.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:28 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:28.871+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:57.978802+0000 front 2024-09-16T09:04:57.978701+0000 (oldest deadline 2024-09-16T09:05:20.278576+0000) 2024-09-16T09:05:29.350 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:28 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:28.922+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:29.350 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:28 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:28.922+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:29.351 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:28 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:28.922+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:29.351 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:28 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:28.922+0000 7f51f2190640 -1 osd.7 767 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.34478.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e767) 2024-09-16T09:05:30.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:29 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:29.873+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:30.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:29 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:29.873+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:30.350 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:29 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:29.873+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:57.978802+0000 front 2024-09-16T09:04:57.978701+0000 (oldest deadline 2024-09-16T09:05:20.278576+0000) 2024-09-16T09:05:30.350 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:29 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:29.926+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:30.350 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:29 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:29.926+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:30.351 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:29 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:29.926+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:30.351 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:29 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:29.926+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6815 osd.3 since back 2024-09-16T09:05:06.013868+0000 front 2024-09-16T09:05:06.013883+0000 (oldest deadline 2024-09-16T09:05:29.513845+0000) 2024-09-16T09:05:30.351 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:29 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:29.926+0000 7f51f2190640 -1 osd.7 767 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.34478.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e767) 2024-09-16T09:05:31.345 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:30 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:30.867+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:42.076943+0000 front 2024-09-16T09:04:42.077025+0000 (oldest deadline 2024-09-16T09:05:07.977060+0000) 2024-09-16T09:05:31.346 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:30 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:30.867+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:47.977655+0000 front 2024-09-16T09:04:47.977623+0000 (oldest deadline 2024-09-16T09:05:13.877633+0000) 2024-09-16T09:05:31.346 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:30 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6[49831]: 2024-09-16T09:05:30.867+0000 7ff8f769f640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:57.978802+0000 front 2024-09-16T09:04:57.978701+0000 (oldest deadline 2024-09-16T09:05:20.278576+0000) 2024-09-16T09:05:31.346 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:30 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:30.889+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:31.346 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:30 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:30.889+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:31.346 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:30 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:30.889+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:31.346 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:30 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:30.889+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6815 osd.3 since back 2024-09-16T09:05:06.013868+0000 front 2024-09-16T09:05:06.013883+0000 (oldest deadline 2024-09-16T09:05:29.513845+0000) 2024-09-16T09:05:31.346 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:30 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:30.889+0000 7f51f2190640 -1 osd.7 767 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.34478.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e767) 2024-09-16T09:05:31.600 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:31 smithi169 podman[68906]: 2024-09-16 09:05:31.345104466 +0000 UTC m=+5.587922362 container died 1ce47534ad745a9c68b7ea7ef9975d5cc041350bfc8c639d6f04cbe2b53bf62f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-16T09:05:31.965 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:31 smithi169 podman[68906]: 2024-09-16 09:05:31.705599476 +0000 UTC m=+5.948417378 container cleanup 1ce47534ad745a9c68b7ea7ef9975d5cc041350bfc8c639d6f04cbe2b53bf62f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:05:31.965 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:31 smithi169 bash[68906]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6 2024-09-16T09:05:31.966 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:31 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:31.859+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:31.966 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:31 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:31.859+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:31.966 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:31 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:31.859+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:31.966 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:31 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:31.859+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6815 osd.3 since back 2024-09-16T09:05:06.013868+0000 front 2024-09-16T09:05:06.013883+0000 (oldest deadline 2024-09-16T09:05:29.513845+0000) 2024-09-16T09:05:31.966 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:31 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:31.859+0000 7f51f2190640 -1 osd.7 767 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.34478.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e767) 2024-09-16T09:05:32.282 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:31 smithi169 podman[68927]: 2024-09-16 09:05:31.965085619 +0000 UTC m=+0.620454940 container remove 1ce47534ad745a9c68b7ea7ef9975d5cc041350bfc8c639d6f04cbe2b53bf62f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T09:05:32.600 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:32 smithi169 podman[69021]: 2024-09-16 09:05:32.184516508 +0000 UTC m=+0.016672201 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:05:32.600 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:32 smithi169 podman[69021]: 2024-09-16 09:05:32.299283355 +0000 UTC m=+0.131439041 container create d6f4a5af32642697137ad0f0d758edfa370ab5a7a3d4943f7b641a8748d9f813 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:05:32.600 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:32 smithi169 podman[69021]: 2024-09-16 09:05:32.510652358 +0000 UTC m=+0.342808052 container init d6f4a5af32642697137ad0f0d758edfa370ab5a7a3d4943f7b641a8748d9f813 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-16T09:05:32.600 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:32 smithi169 podman[69021]: 2024-09-16 09:05:32.515428681 +0000 UTC m=+0.347584365 container start d6f4a5af32642697137ad0f0d758edfa370ab5a7a3d4943f7b641a8748d9f813 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-16T09:05:33.056 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:32 smithi169 podman[69021]: 2024-09-16 09:05:32.601602192 +0000 UTC m=+0.433757881 container attach d6f4a5af32642697137ad0f0d758edfa370ab5a7a3d4943f7b641a8748d9f813 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:05:33.056 INFO:journalctl@ceph.osd.6.smithi169.stdout:Sep 16 09:05:32 smithi169 podman[69021]: 2024-09-16 09:05:32.702758215 +0000 UTC m=+0.534913899 container died d6f4a5af32642697137ad0f0d758edfa370ab5a7a3d4943f7b641a8748d9f813 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731) 2024-09-16T09:05:33.056 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:32 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:32.819+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:33.057 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:32 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:32.819+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:33.057 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:32 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:32.819+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:33.057 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:32 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:32.819+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6815 osd.3 since back 2024-09-16T09:05:06.013868+0000 front 2024-09-16T09:05:06.013883+0000 (oldest deadline 2024-09-16T09:05:29.513845+0000) 2024-09-16T09:05:33.057 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:32 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:32.819+0000 7f51f2190640 -1 osd.7 767 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.34478.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e767) 2024-09-16T09:05:33.381 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.6.service' 2024-09-16T09:05:33.420 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:05:33.421 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-16T09:05:33.421 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-16T09:05:33.421 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.7 2024-09-16T09:05:33.745 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:33 smithi169 systemd[1]: Stopping Ceph osd.7 for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:05:34.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:33 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:33.743+0000 7f51f9b83640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:05:34.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:33 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:33.743+0000 7f51f9b83640 -1 osd.7 767 *** Got signal Terminated *** 2024-09-16T09:05:34.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:33 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:33.743+0000 7f51f9b83640 -1 osd.7 767 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:05:34.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:33 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:33.811+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:34.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:33 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:33.811+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:34.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:33 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:33.811+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:34.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:33 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:33.811+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6815 osd.3 since back 2024-09-16T09:05:06.013868+0000 front 2024-09-16T09:05:06.013883+0000 (oldest deadline 2024-09-16T09:05:29.513845+0000) 2024-09-16T09:05:34.101 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:33 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:33.811+0000 7f51f2190640 -1 osd.7 767 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.34478.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e767) 2024-09-16T09:05:35.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:34 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:34.765+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:35.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:34 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:34.765+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:35.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:34 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:34.765+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:35.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:34 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:34.765+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6815 osd.3 since back 2024-09-16T09:05:06.013868+0000 front 2024-09-16T09:05:06.013883+0000 (oldest deadline 2024-09-16T09:05:29.513845+0000) 2024-09-16T09:05:35.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:34 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:34.765+0000 7f51f2190640 -1 osd.7 767 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.34478.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e767) 2024-09-16T09:05:36.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:35.813+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:36.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:35.813+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:36.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:35.813+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:36.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:35.813+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6815 osd.3 since back 2024-09-16T09:05:06.013868+0000 front 2024-09-16T09:05:06.013883+0000 (oldest deadline 2024-09-16T09:05:29.513845+0000) 2024-09-16T09:05:36.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:35 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:35.813+0000 7f51f2190640 -1 osd.7 767 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.34478.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e767) 2024-09-16T09:05:37.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:36 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:36.834+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:37.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:36 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:36.834+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:37.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:36 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:36.834+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:37.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:36 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:36.834+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6815 osd.3 since back 2024-09-16T09:05:06.013868+0000 front 2024-09-16T09:05:06.013883+0000 (oldest deadline 2024-09-16T09:05:29.513845+0000) 2024-09-16T09:05:37.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:36 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:36.834+0000 7f51f2190640 -1 osd.7 767 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.34478.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e767) 2024-09-16T09:05:37.359 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:37 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:05:37.020Z caller=dispatch.go:353 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"\": dial tcp 172.21.15.169:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"\": dial tcp 172.21.15.160:8443: connect: connection refused" 2024-09-16T09:05:37.359 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:37 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:05:37.021Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.160:8443: connect: connection refused" 2024-09-16T09:05:37.359 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:37 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:05:37.021Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.169:8443: connect: connection refused" 2024-09-16T09:05:38.350 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:37 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:37.867+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6803 osd.0 since back 2024-09-16T09:04:40.811043+0000 front 2024-09-16T09:04:40.811131+0000 (oldest deadline 2024-09-16T09:05:04.910942+0000) 2024-09-16T09:05:38.350 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:37 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:37.867+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6807 osd.1 since back 2024-09-16T09:04:50.811966+0000 front 2024-09-16T09:04:50.811978+0000 (oldest deadline 2024-09-16T09:05:13.111888+0000) 2024-09-16T09:05:38.350 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:37 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:37.867+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6811 osd.2 since back 2024-09-16T09:04:55.412956+0000 front 2024-09-16T09:04:55.412914+0000 (oldest deadline 2024-09-16T09:05:20.712926+0000) 2024-09-16T09:05:38.350 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:37 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:37.867+0000 7f51f2190640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.160:6815 osd.3 since back 2024-09-16T09:05:06.013868+0000 front 2024-09-16T09:05:06.013883+0000 (oldest deadline 2024-09-16T09:05:29.513845+0000) 2024-09-16T09:05:38.350 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:37 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7[56099]: 2024-09-16T09:05:37.867+0000 7f51f2190640 -1 osd.7 767 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.34478.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e767) 2024-09-16T09:05:39.600 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:39 smithi169 podman[69147]: 2024-09-16 09:05:39.308040659 +0000 UTC m=+5.670916514 container died 6c1212974f22a76e1b2c0fa8b9d9fa15f52f0a86e9ad102e71c96822993d6ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-16T09:05:40.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:39 smithi169 podman[69147]: 2024-09-16 09:05:39.632604065 +0000 UTC m=+5.995479909 container cleanup 6c1212974f22a76e1b2c0fa8b9d9fa15f52f0a86e9ad102e71c96822993d6ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:05:40.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:39 smithi169 bash[69147]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7 2024-09-16T09:05:40.100 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:39 smithi169 podman[69180]: 2024-09-16 09:05:39.809048416 +0000 UTC m=+0.502058110 container remove 6c1212974f22a76e1b2c0fa8b9d9fa15f52f0a86e9ad102e71c96822993d6ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:05:40.537 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:40 smithi169 podman[69272]: 2024-09-16 09:05:40.149323383 +0000 UTC m=+0.016279302 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:05:40.537 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:40 smithi169 podman[69272]: 2024-09-16 09:05:40.268002248 +0000 UTC m=+0.134958157 container create 627304ad45a755e208b5c3ce51f2d6a88c5e700aeab72d7b65b80f9e7f6f380f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:05:40.537 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:40 smithi169 podman[69272]: 2024-09-16 09:05:40.453732796 +0000 UTC m=+0.320688706 container init 627304ad45a755e208b5c3ce51f2d6a88c5e700aeab72d7b65b80f9e7f6f380f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-16T09:05:40.538 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:40 smithi169 podman[69272]: 2024-09-16 09:05:40.458731382 +0000 UTC m=+0.325687292 container start 627304ad45a755e208b5c3ce51f2d6a88c5e700aeab72d7b65b80f9e7f6f380f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-16T09:05:40.538 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:40 smithi169 podman[69272]: 2024-09-16 09:05:40.536773024 +0000 UTC m=+0.403728943 container attach 627304ad45a755e208b5c3ce51f2d6a88c5e700aeab72d7b65b80f9e7f6f380f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:05:40.850 INFO:journalctl@ceph.osd.7.smithi169.stdout:Sep 16 09:05:40 smithi169 podman[69272]: 2024-09-16 09:05:40.644177099 +0000 UTC m=+0.511133022 container died 627304ad45a755e208b5c3ce51f2d6a88c5e700aeab72d7b65b80f9e7f6f380f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T09:05:41.254 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@osd.7.service' 2024-09-16T09:05:41.296 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:05:41.296 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-16T09:05:41.296 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-16T09:05:41.296 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-27cca2fc-7407-11ef-bceb-c7b262605968@rgw.foo.a 2024-09-16T09:05:41.617 INFO:journalctl@ceph.rgw.foo.a.smithi160.stdout:Sep 16 09:05:41 smithi160 systemd[1]: Stopping Ceph rgw.foo.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:05:41.947 INFO:journalctl@ceph.rgw.foo.a.smithi160.stdout:Sep 16 09:05:41 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-rgw-foo-a[66241]: 2024-09-16T09:05:41.617+0000 7f50967e3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:05:41.948 INFO:journalctl@ceph.rgw.foo.a.smithi160.stdout:Sep 16 09:05:41 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-rgw-foo-a[66241]: 2024-09-16T09:05:41.617+0000 7f509a04f880 -1 shutting down 2024-09-16T09:05:47.448 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:05:47.021Z caller=dispatch.go:353 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.169:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.160:8443: connect: connection refused" 2024-09-16T09:05:47.448 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:05:47.021Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.160:8443: connect: connection refused" 2024-09-16T09:05:47.448 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:47 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:05:47.022Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.169:8443: connect: connection refused" 2024-09-16T09:05:51.941 INFO:journalctl@ceph.rgw.foo.a.smithi160.stdout:Sep 16 09:05:51 smithi160 bash[148209]: time="2024-09-16T09:05:51Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-27cca2fc-7407-11ef-bceb-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-16T09:05:51.941 INFO:journalctl@ceph.rgw.foo.a.smithi160.stdout:Sep 16 09:05:51 smithi160 podman[148209]: 2024-09-16 09:05:51.638220395 +0000 UTC m=+10.137624160 container died 2591e5b22df76127c612bbd6641334c0c32f421dafdd3282c4cfa2d6c80d2c59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-rgw-foo-a, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T09:05:52.198 INFO:journalctl@ceph.rgw.foo.a.smithi160.stdout:Sep 16 09:05:52 smithi160 podman[148209]: 2024-09-16 09:05:52.047393121 +0000 UTC m=+10.546796885 container cleanup 2591e5b22df76127c612bbd6641334c0c32f421dafdd3282c4cfa2d6c80d2c59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:05:52.198 INFO:journalctl@ceph.rgw.foo.a.smithi160.stdout:Sep 16 09:05:52 smithi160 bash[148209]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-rgw-foo-a 2024-09-16T09:05:52.487 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@rgw.foo.a.service' 2024-09-16T09:05:52.521 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:05:52.522 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-16T09:05:52.522 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-16T09:05:52.522 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-27cca2fc-7407-11ef-bceb-c7b262605968@prometheus.a 2024-09-16T09:05:52.823 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:52 smithi169 systemd[1]: Stopping Ceph prometheus.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:05:53.081 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:52 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:52.822Z caller=main.go:964 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-16T09:05:53.081 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:52 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:52.823Z caller=main.go:988 level=info msg="Stopping scrape discovery manager..." 2024-09-16T09:05:53.081 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:52 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:52.823Z caller=main.go:1002 level=info msg="Stopping notify discovery manager..." 2024-09-16T09:05:53.081 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:52 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:52.823Z caller=manager.go:177 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-16T09:05:53.081 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:52 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:52.823Z caller=main.go:984 level=info msg="Scrape discovery manager stopped" 2024-09-16T09:05:53.081 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:52 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:52.823Z caller=main.go:998 level=info msg="Notify discovery manager stopped" 2024-09-16T09:05:53.082 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:52 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:52.823Z caller=manager.go:187 level=info component="rule manager" msg="Rule manager stopped" 2024-09-16T09:05:53.082 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:52 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:52.823Z caller=main.go:1039 level=info msg="Stopping scrape manager..." 2024-09-16T09:05:53.082 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:52 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:52.823Z caller=main.go:1031 level=info msg="Scrape manager stopped" 2024-09-16T09:05:53.082 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:52 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:52.824Z caller=notifier.go:618 level=info component=notifier msg="Stopping notification manager..." 2024-09-16T09:05:53.082 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:52 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:52.824Z caller=main.go:1261 level=info msg="Notifier manager stopped" 2024-09-16T09:05:53.082 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:52 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a[64762]: ts=2024-09-16T09:05:52.824Z caller=main.go:1273 level=info msg="See you next time!" 2024-09-16T09:05:53.082 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:52 smithi169 podman[69400]: 2024-09-16 09:05:52.833143044 +0000 UTC m=+0.114734394 container died 45a00bae074a7d5e0d5dee764328e8c530b90aed2f2c41173a382faac3581ea8 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T09:05:53.350 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:53 smithi169 podman[69400]: 2024-09-16 09:05:53.187091098 +0000 UTC m=+0.468682447 container cleanup 45a00bae074a7d5e0d5dee764328e8c530b90aed2f2c41173a382faac3581ea8 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T09:05:53.350 INFO:journalctl@ceph.prometheus.a.smithi169.stdout:Sep 16 09:05:53 smithi169 bash[69400]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-prometheus-a 2024-09-16T09:05:53.613 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-27cca2fc-7407-11ef-bceb-c7b262605968@prometheus.a.service' 2024-09-16T09:05:53.647 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:05:53.648 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-16T09:05:53.648 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm rm-cluster --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T09:05:53.849 INFO:teuthology.orchestra.run.smithi160.stdout:Deleting cluster with fsid: 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T09:05:55.698 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:55 smithi160 systemd[1]: Stopping Ceph alertmanager.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:05:56.079 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:55 smithi160 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-16T09:05:55.796Z caller=main.go:542 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-16T09:05:56.079 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:55 smithi160 podman[148719]: 2024-09-16 09:05:55.836960639 +0000 UTC m=+0.156640516 container died 6156f45f8ef04943f69f4620eba3a5e176ba24abf2820c0fdc369d7a5e09d162 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T09:05:56.345 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:56 smithi160 podman[148719]: 2024-09-16 09:05:56.168943564 +0000 UTC m=+0.488623438 container cleanup 6156f45f8ef04943f69f4620eba3a5e176ba24abf2820c0fdc369d7a5e09d162 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T09:05:56.345 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:56 smithi160 bash[148719]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a 2024-09-16T09:05:56.626 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:56 smithi160 podman[148736]: 2024-09-16 09:05:56.345137599 +0000 UTC m=+0.502884424 container remove 6156f45f8ef04943f69f4620eba3a5e176ba24abf2820c0fdc369d7a5e09d162 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T09:05:56.626 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:56 smithi160 podman[148736]: 2024-09-16 09:05:56.420628963 +0000 UTC m=+0.578375744 volume remove 6c32d6cf87daf28a6ca3640e0f9c8987224e083ee6b970bc1215fb34a23d73d9 2024-09-16T09:05:56.948 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 09:05:56 smithi160 systemd[1]: Stopping Ceph node-exporter.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:05:56.948 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:56 smithi160 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-16T09:05:56.948 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:56 smithi160 systemd[1]: Stopped Ceph alertmanager.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T09:05:56.948 INFO:journalctl@ceph.alertmanager.a.smithi160.stdout:Sep 16 09:05:56 smithi160 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@alertmanager.a.service: Consumed 1.916s CPU time. 2024-09-16T09:05:57.332 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 09:05:57 smithi160 podman[148857]: 2024-09-16 09:05:57.077405493 +0000 UTC m=+0.110768773 container died 88d23ccb71c1cd7f91ff4e42b02f7e4ab24d2b1a75cbd28aca32d67d88970127 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T09:05:57.698 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 09:05:57 smithi160 podman[148857]: 2024-09-16 09:05:57.431373551 +0000 UTC m=+0.464736815 container cleanup 88d23ccb71c1cd7f91ff4e42b02f7e4ab24d2b1a75cbd28aca32d67d88970127 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T09:05:57.698 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 09:05:57 smithi160 bash[148857]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a 2024-09-16T09:05:57.990 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 09:05:57 smithi160 podman[148875]: 2024-09-16 09:05:57.699417042 +0000 UTC m=+0.625817267 container remove 88d23ccb71c1cd7f91ff4e42b02f7e4ab24d2b1a75cbd28aca32d67d88970127 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T09:05:57.991 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 09:05:57 smithi160 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-16T09:05:57.991 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 09:05:57 smithi160 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-16T09:05:57.991 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 09:05:57 smithi160 systemd[1]: Stopped Ceph node-exporter.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T09:05:57.991 INFO:journalctl@ceph.node-exporter.a.smithi160.stdout:Sep 16 09:05:57 smithi160 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@node-exporter.a.service: Consumed 3.612s CPU time. 2024-09-16T09:05:58.462 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm rm-cluster --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T09:05:58.663 INFO:teuthology.orchestra.run.smithi169.stdout:Deleting cluster with fsid: 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T09:06:00.350 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:05:59 smithi169 systemd[1]: Stopping Ceph iscsi.iscsi.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:06:00.350 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:06:00 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a[60829]: debug Shutdown received 2024-09-16T09:06:10.455 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:06:10 smithi169 bash[69816]: time="2024-09-16T09:06:10Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-16T09:06:10.455 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:06:10 smithi169 podman[69816]: 2024-09-16 09:06:10.169113146 +0000 UTC m=+10.116138517 container died 4f95bb703dd63dc4d64177962172c1812c2f3ddae0237c3d6efa94fd1eaad0e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:06:10.840 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:06:10 smithi169 podman[69816]: 2024-09-16 09:06:10.55432561 +0000 UTC m=+10.501350980 container cleanup 4f95bb703dd63dc4d64177962172c1812c2f3ddae0237c3d6efa94fd1eaad0e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:06:10.840 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:06:10 smithi169 bash[69816]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a 2024-09-16T09:06:11.100 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:06:10 smithi169 podman[69840]: 2024-09-16 09:06:10.839465927 +0000 UTC m=+0.669685068 container remove 4f95bb703dd63dc4d64177962172c1812c2f3ddae0237c3d6efa94fd1eaad0e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-iscsi-iscsi-a, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-16T09:06:11.100 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:06:10 smithi169 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-16T09:06:11.100 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:06:11 smithi169 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-16T09:06:11.100 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:06:11 smithi169 systemd[1]: Stopped Ceph iscsi.iscsi.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T09:06:11.101 INFO:journalctl@ceph.iscsi.iscsi.a.smithi169.stdout:Sep 16 09:06:11 smithi169 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Consumed 2.788s CPU time. 2024-09-16T09:06:12.600 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 09:06:12 smithi169 systemd[1]: Stopping Ceph grafana.a for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:06:12.979 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 09:06:12 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=server t=2024-09-16T09:06:12.713167791Z level=info msg="Shutdown started" reason="System signal: terminated" 2024-09-16T09:06:12.979 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 09:06:12 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=tracing t=2024-09-16T09:06:12.713433578Z level=info msg="Closing tracing" 2024-09-16T09:06:12.979 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 09:06:12 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=ticker t=2024-09-16T09:06:12.71344252Z level=info msg=stopped last_tick=2024-09-16T09:06:10Z 2024-09-16T09:06:12.980 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 09:06:12 smithi169 ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a[63486]: logger=grafana-apiserver t=2024-09-16T09:06:12.713460618Z level=info msg="StorageObjectCountTracker pruner is exiting" 2024-09-16T09:06:12.980 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 09:06:12 smithi169 podman[70127]: 2024-09-16 09:06:12.72365414 +0000 UTC m=+0.117958145 container died 9af72bc33191d0a6d739b3ce68288a88c0a27350667b293f99a67b2429cd05bc (image=quay.io/ceph/grafana:10.4.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-16T09:06:13.329 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 09:06:13 smithi169 podman[70127]: 2024-09-16 09:06:13.068803998 +0000 UTC m=+0.463108010 container cleanup 9af72bc33191d0a6d739b3ce68288a88c0a27350667b293f99a67b2429cd05bc (image=quay.io/ceph/grafana:10.4.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-16T09:06:13.329 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 09:06:13 smithi169 bash[70127]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a 2024-09-16T09:06:13.600 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 09:06:13 smithi169 podman[70144]: 2024-09-16 09:06:13.32852578 +0000 UTC m=+0.604950204 container remove 9af72bc33191d0a6d739b3ce68288a88c0a27350667b293f99a67b2429cd05bc (image=quay.io/ceph/grafana:10.4.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-16T09:06:13.600 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 09:06:13 smithi169 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@grafana.a.service: Deactivated successfully. 2024-09-16T09:06:13.600 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 09:06:13 smithi169 systemd[1]: Stopped Ceph grafana.a for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T09:06:13.600 INFO:journalctl@ceph.grafana.a.smithi169.stdout:Sep 16 09:06:13 smithi169 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@grafana.a.service: Consumed 12.746s CPU time. 2024-09-16T09:06:13.946 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 09:06:13 smithi169 systemd[1]: Stopping Ceph node-exporter.b for 27cca2fc-7407-11ef-bceb-c7b262605968... 2024-09-16T09:06:14.225 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 09:06:13 smithi169 podman[70265]: 2024-09-16 09:06:13.945047892 +0000 UTC m=+0.114616187 container died 8b16f01ba1fe7784d0c5df1548d6b86c05eab1e542adadc763add53a6fffd305 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T09:06:14.567 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 09:06:14 smithi169 podman[70265]: 2024-09-16 09:06:14.315605 +0000 UTC m=+0.485173298 container cleanup 8b16f01ba1fe7784d0c5df1548d6b86c05eab1e542adadc763add53a6fffd305 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T09:06:14.567 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 09:06:14 smithi169 bash[70265]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b 2024-09-16T09:06:14.850 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 09:06:14 smithi169 podman[70281]: 2024-09-16 09:06:14.566741412 +0000 UTC m=+0.625763319 container remove 8b16f01ba1fe7784d0c5df1548d6b86c05eab1e542adadc763add53a6fffd305 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-27cca2fc-7407-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T09:06:14.850 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 09:06:14 smithi169 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-16T09:06:14.850 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 09:06:14 smithi169 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-16T09:06:14.850 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 09:06:14 smithi169 systemd[1]: Stopped Ceph node-exporter.b for 27cca2fc-7407-11ef-bceb-c7b262605968. 2024-09-16T09:06:14.850 INFO:journalctl@ceph.node-exporter.b.smithi169.stdout:Sep 16 09:06:14 smithi169 systemd[1]: ceph-27cca2fc-7407-11ef-bceb-c7b262605968@node-exporter.b.service: Consumed 3.635s CPU time. 2024-09-16T09:06:15.344 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T09:06:15.378 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T09:06:15.405 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-16T09:06:15.407 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906962/remote/smithi160/crash 2024-09-16T09:06:15.408 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/crash -- . 2024-09-16T09:06:15.478 INFO:teuthology.orchestra.run.smithi160.stderr:tar: /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T09:06:15.478 INFO:teuthology.orchestra.run.smithi160.stderr:tar: Error is not recoverable: exiting now 2024-09-16T09:06:15.480 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906962/remote/smithi169/crash 2024-09-16T09:06:15.481 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/crash -- . 2024-09-16T09:06:15.516 INFO:teuthology.orchestra.run.smithi169.stderr:tar: /var/lib/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T09:06:15.516 INFO:teuthology.orchestra.run.smithi169.stderr:tar: Error is not recoverable: exiting now 2024-09-16T09:06:15.517 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-16T09:06:15.517 DEBUG:teuthology.orchestra.run.smithi160:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v CEPHADM_STRAY_DAEMON | head -n 1 2024-09-16T09:06:15.551 INFO:tasks.cephadm:Compressing logs... 2024-09-16T09:06:15.551 DEBUG:teuthology.orchestra.run.smithi160:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T09:06:15.594 DEBUG:teuthology.orchestra.run.smithi169:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T09:06:15.617 INFO:teuthology.orchestra.run.smithi160.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-16T09:06:15.618 INFO:teuthology.orchestra.run.smithi160.stderr:: No such file or directory 2024-09-16T09:06:15.618 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-16T09:06:15.618 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T09:06:15.624 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T09:06:15.624 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-16T09:06:15.624 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T09:06:15.624 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-16T09:06:15.624 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-16T09:06:15.625 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-mon.a.log: 89.3% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.cephadm.log.gzgzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-16T09:06:15.625 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-16T09:06:15.625 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.log 2024-09-16T09:06:15.627 INFO:teuthology.orchestra.run.smithi169.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/tcmu-runner.log 2024-09-16T09:06:15.627 INFO:teuthology.orchestra.run.smithi169.stderr:: No such file or directory 2024-09-16T09:06:15.627 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-16T09:06:15.627 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose --/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/tcmu-runner.log: /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T09:06:15.627 INFO:teuthology.orchestra.run.smithi169.stderr: 62.9% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/tcmu-runner.log.gz 2024-09-16T09:06:15.627 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-16T09:06:15.627 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T09:06:15.627 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T09:06:15.628 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-16T09:06:15.628 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-16T09:06:15.628 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 79.8% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T09:06:15.628 INFO:teuthology.orchestra.run.smithi169.stderr: --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.log 2024-09-16T09:06:15.628 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-16T09:06:15.628 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-16T09:06:15.628 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-mgr.x.log: 87.2% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T09:06:15.628 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T09:06:15.630 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-mon.b.log: 92.2% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T09:06:15.633 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-16T09:06:15.634 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.2.log: /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- 91.3% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T09:06:15.635 INFO:teuthology.orchestra.run.smithi160.stderr: /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-16T09:06:15.635 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-16T09:06:15.635 INFO:teuthology.orchestra.run.smithi169.stderr: 91.2% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T09:06:15.637 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-client.rgw.foo.a.log: 95.2% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T09:06:15.641 INFO:teuthology.orchestra.run.smithi160.stderr: 92.8% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T09:06:15.641 INFO:teuthology.orchestra.run.smithi169.stderr: 91.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T09:06:15.643 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T09:06:15.674 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 91.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T09:06:15.677 INFO:teuthology.orchestra.run.smithi160.stderr: 93.1% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-16T09:06:16.042 INFO:teuthology.orchestra.run.smithi160.stderr: 90.2% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-16T09:06:16.762 INFO:teuthology.orchestra.run.smithi169.stderr: 92.6% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-16T09:06:17.007 INFO:teuthology.orchestra.run.smithi160.stderr: 92.3% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-16T09:06:17.615 INFO:teuthology.orchestra.run.smithi160.stderr: 91.8% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-16T09:06:21.804 INFO:teuthology.orchestra.run.smithi169.stderr: 93.8% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-16T09:06:22.079 INFO:teuthology.orchestra.run.smithi169.stderr: 93.8% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-16T09:06:22.480 INFO:teuthology.orchestra.run.smithi169.stderr: 93.8% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-16T09:06:22.512 INFO:teuthology.orchestra.run.smithi160.stderr: 93.9% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-16T09:06:22.537 INFO:teuthology.orchestra.run.smithi160.stderr: 93.8% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-16T09:06:22.681 INFO:teuthology.orchestra.run.smithi169.stderr: 93.9% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-16T09:06:22.683 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-09-16T09:06:22.683 INFO:teuthology.orchestra.run.smithi169.stderr:real 0m7.075s 2024-09-16T09:06:22.683 INFO:teuthology.orchestra.run.smithi169.stderr:user 0m26.788s 2024-09-16T09:06:22.683 INFO:teuthology.orchestra.run.smithi169.stderr:sys 0m0.936s 2024-09-16T09:06:23.133 INFO:teuthology.orchestra.run.smithi160.stderr: 93.9% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-16T09:06:23.185 INFO:teuthology.orchestra.run.smithi160.stderr: 94.0% -- replaced with /var/log/ceph/27cca2fc-7407-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-16T09:06:23.187 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-16T09:06:23.188 INFO:teuthology.orchestra.run.smithi160.stderr:real 0m7.581s 2024-09-16T09:06:23.188 INFO:teuthology.orchestra.run.smithi160.stderr:user 0m31.515s 2024-09-16T09:06:23.188 INFO:teuthology.orchestra.run.smithi160.stderr:sys 0m1.109s 2024-09-16T09:06:23.188 INFO:tasks.cephadm:Archiving logs... 2024-09-16T09:06:23.188 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906962/remote/smithi160/log 2024-09-16T09:06:23.189 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T09:06:27.377 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906962/remote/smithi169/log 2024-09-16T09:06:27.378 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T09:06:31.174 INFO:tasks.cephadm:Removing cluster... 2024-09-16T09:06:31.174 DEBUG:teuthology.orchestra.run.smithi160:> sudo cephadm rm-cluster --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 --force 2024-09-16T09:06:31.382 INFO:teuthology.orchestra.run.smithi160.stdout:Deleting cluster with fsid: 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T09:06:31.674 DEBUG:teuthology.orchestra.run.smithi169:> sudo cephadm rm-cluster --fsid 27cca2fc-7407-11ef-bceb-c7b262605968 --force 2024-09-16T09:06:31.881 INFO:teuthology.orchestra.run.smithi169.stdout:Deleting cluster with fsid: 27cca2fc-7407-11ef-bceb-c7b262605968 2024-09-16T09:06:32.173 INFO:tasks.cephadm:Teardown complete 2024-09-16T09:06:32.174 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T09:06:32.185 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T09:06:32.185 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T09:06:32.188 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T09:06:32.268 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T09:06:32.277 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T09:06:32.277 DEBUG:teuthology.orchestra.run.smithi160:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T09:06:32.280 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T09:06:32.292 INFO:teuthology.orchestra.run.smithi160.stderr:bash: line 1: ntpq: command not found 2024-09-16T09:06:32.294 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-09-16T09:06:32.296 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T09:06:32.297 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-09-16T09:06:32.297 INFO:teuthology.orchestra.run.smithi160.stdout:^* hv01.front.sepia.ceph.com 3 6 377 51 +91us[ +142us] +/- 52ms 2024-09-16T09:06:32.297 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 53 -528us[ -478us] +/- 72ms 2024-09-16T09:06:32.297 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 +359us[ +359us] +/- 63ms 2024-09-16T09:06:32.297 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T09:06:32.299 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T09:06:32.299 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-09-16T09:06:32.299 INFO:teuthology.orchestra.run.smithi169.stdout:^* hv01.front.sepia.ceph.com 3 6 377 51 +97us[ +60us] +/- 52ms 2024-09-16T09:06:32.300 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 52 -511us[ -548us] +/- 72ms 2024-09-16T09:06:32.300 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 +374us[ +337us] +/- 63ms 2024-09-16T09:06:32.300 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T09:06:32.300 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T09:06:32.309 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T09:06:32.310 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T09:06:32.319 DEBUG:teuthology.orchestra.run.smithi160:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T09:06:32.410 DEBUG:teuthology.orchestra.run.smithi169:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T09:06:32.481 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-16T09:06:32.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:06:32.510 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-16T09:06:32.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:06:32.538 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T09:06:32.548 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T09:06:32.556 INFO:teuthology.task.internal:Duration was 2030.063375 seconds 2024-09-16T09:06:32.558 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T09:06:32.567 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T09:06:32.567 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T09:06:32.569 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T09:06:32.606 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T09:06:32.617 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T09:06:32.986 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T09:06:32.986 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi160.front.sepia.ceph.com 2024-09-16T09:06:32.986 DEBUG:teuthology.orchestra.run.smithi160:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T09:06:33.047 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2024-09-16T09:06:33.048 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T09:06:33.070 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T09:06:33.071 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T09:06:33.090 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T09:06:33.137 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T09:06:33.137 DEBUG:teuthology.orchestra.run.smithi160:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T09:06:33.158 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T09:06:33.553 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T09:06:33.562 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T09:06:33.562 DEBUG:teuthology.orchestra.run.smithi160:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T09:06:33.587 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T09:06:33.612 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T09:06:33.621 DEBUG:teuthology.orchestra.run.smithi160:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T09:06:33.630 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T09:06:33.650 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = core 2024-09-16T09:06:33.675 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2024-09-16T09:06:33.722 DEBUG:teuthology.orchestra.run.smithi160:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T09:06:33.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:06:33.736 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T09:06:33.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:06:33.776 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T09:06:33.786 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T09:06:33.787 DEBUG:teuthology.misc:Transferring archived files from smithi160:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906962/remote/smithi160 2024-09-16T09:06:33.787 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T09:06:33.848 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906962/remote/smithi169 2024-09-16T09:06:33.849 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T09:06:33.887 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T09:06:33.888 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T09:06:33.890 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T09:06:33.936 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T09:06:33.949 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T09:06:33.949 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T09:06:33.958 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T09:06:33.958 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T09:06:33.960 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T09:06:33.973 INFO:teuthology.orchestra.run.smithi160.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 09:06 /home/ubuntu/cephtest 2024-09-16T09:06:33.991 INFO:teuthology.orchestra.run.smithi169.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 09:06 /home/ubuntu/cephtest 2024-09-16T09:06:33.992 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T09:06:34.002 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T09:06:34.024 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} duration: 2030.0633754730225 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-16T09:06:34.024 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T09:06:34.101 INFO:teuthology.run:pass